training effectiveness and transfer -- a mixed methods investigation

8/12/2019 Training Effectiveness and Transfer -- A Mixed Methods Investigation

1/22

Training Effectiveness and Transfer: A Mixed Methods Investigation

Dr Mary C. CowmanDepartment of Health, Sport and Exercise Science

Waterford Institute of TechnologyWaterfordIRELAND

Email: [email protected]

Dr Alma M. McCarthyDepartment of Management

J.E Cairnes Graduate School of Business & Public PolicyNational University of Ireland, Galway

Galway

IRELANDEmail: [email protected]

Stream: HRD, Evaluation and Learning


2/22

Abstract

The effectiveness of workplace training and development is largely determined bythe extent to which training transfer occurs and is sustained over time. However,there are gaps in our understanding of the training transfer process (Burke and

Hutchins, 2008; Holton, Chen and Naquin, 2003; Martin, 2010; Spitzer, 2005). Usingboth quantitative and qualitative data, this paper investigates the impact of a trainingand development intervention at individual and organisational level in a health carecontext in Ireland. Furthermore, it examines organisational system factors affectingtraining transfer post training.

Keywords

Training Transfer; evaluation; training effectiveness; training impact;, mixedmethods; organisational system factors


3/22

Training Effectiveness and Transfer: A Mixed Methods Investigation

1.0 Introduction

The effectiveness of workplace training and development is largely determined bythe extent to which training transfer occurs and is sustained over time. However,there are gaps in our undertanding of the training transfer process (Burke andHutchins, 2008; Holton, Chen and Naquin, 2003; Martin, 2010; Spitzer, 2005).Ruona, Leimbach, Holton and Bates (2002) suggest that methods to evaluate theimpact of training are inadequate for answering many of the questions regarding theeffectiveness of training and development efforts. These factors have implicationsfor the extent to which training transfer research is contributing to our understandingof the process of training transfer and effectiveness. Using both quantitative andqualitative data, this paper investigates the impact of a training and developmentintervention at individual and organisational level in a health care context in Ireland.

Furthermore, it examines organisational system factors affecting training transferpost training.

2.0 Training Evaluation and Effectiveness

Measuring and evaluating the effectiveness of training and developmentprogrammes is one of the most critical components of SHRD (Horwitz 1999).However, according to Hutchins and Burke (2007), studies have estimated that 85%of training resources are dedicated to designing and delivering training with theremaining 15% divided between front-end analysis and evaluation activities. Wheremeasurement of training effectiveness is taking place, Spitzer (2005) argues that

HRD is failing to demonstrate that investment in training and developmentprogrammes is producing results at organisational level.

Defining training effectiveness is complex which has implications for thedevelopment of strategies to measure training effectiveness. Alvarez, Salas andGarofano (2004) distinguish between the term training evaluation and trainingeffectiveness. The former is described as a measurement technique to determine iftraining goals have been met. The latter is a theoretical approach used to study theindividual, training and organisational variables that are likely to influence trainingoutcomes.

Previous training evaluation studies have been criticised for the lack of informationprovided on the impact of the training intervention at both the individual (level three)and the organisational levels (level four) (Cheng & Ho 2001), for the failure to identifysub-categories of learning (Alliger et al. 1997) or, to measure the horizontal impactof training at individual level (Kearns 2005) such as the impact on the team or theunit. Wang and Sun (2009) suggest that the purpose of investing in employees is toenhance their current and future productivity for the organisation. For example,Mayo(2000) suggests that HRD can contribute to employee added value throughincreased motivation, commitment, efficiency and competence.

Techniques for measuring training outcomes proposed by Kearns (2005) andSptizers (2005) are reflective of a training effectiveness rather than a training


4/22

evaluation approach to investigating the impact of training interventions. Thesearguments demonstrate the importance of adopting an approach for measuringtraining effectiveness that is capable of establishing different forms of trainingtransfer, the impact of the training intervention at different levels of investigation andthe manner in which different factors have facilitated or inhibited the transfer

process. According to Wang and Spitzer (2005), the most recent stage in theevolution of measurement and evaluation is characterised by research oriented,practice based comprehensive methodologies which are rooted in existing theoriesand which seek to develop more comprehensive, robust evaluation techniques in linewith the effectiveness approach. Tan, Hall and Boyce (2003) argue that the mostdeveloped and used models focus on defining different training effectiveness criteriaand their organisational implications which reflect the definition of trainingeffectiveness proposed by Alvarez et al (2004).

Using a mixed methods approach, this paper investigates training effectiveness inthe healthcare sector in Ireland. Mixed methods research (the combined use of

quantitative and qualitative methods in the same study) is becoming an increasinglypopular approach in the fields of psychology, education, sociology, management andhealth sciences (Greene, Caracelli and Graham, 1989; OCathain, 2009). The centralpremise for the use of mixed methods is that the use of quantitative and qualitativeapproaches in combination provides greater understanding of research problemsthan monomethod designs (Cresswell and Plano Clark, 2007; Tashakkori andTeddlie, 2003). The model developed for the purpose of this study builds on theevaluation and effectiveness frameworks presented by Rouiller and Goldstein(1993), Cannon-Bowers, Salas, Tannenbaum and Mathieu (1995), Alvarez et al.(2004), Holton (1996 and 2000), and Spitzer (2005). In this study, we exploretraining transfer as am important dimension of training effectiveness in a health caresector. The next section discusses the concept of training transfer and how it can beoperationalised.

3.0 Organisation Transfer System and Training TransferTransfer of training is the application of learned knowledge, skills and attitudes to thejob and subsequent maintenance over time(Cheng & Ho 2001) for the purposes ofimproving the job performance (Velada & Caetano 2007). Geilen (cited in Van derKlink, Gielen, and Nauta 2001) identified three dimensions of transfer: the direction;

the level of complexity; and the distance. The directionof training transfer refers toeither positive transfer where training leads to desired performance or negativetransfer, where it fails to produce intended job performance. Lateral transfer refers tothe learner being able to achieve a task at the same complexity level as the taskalready mastered whereas vertical transfer refers to the ability to apply learning tosimilar or more complex skills. Regarding distance, near transfer refers to training intasks that are similar or equal to the learners job tasks. Far transfer is where thereis a lack of similarity to the job tasks and training focuses on understanding and theapplication of principles or rules. For the purposes of this study, the direction oftraining transfer and the influence of organisational factors on the direction of trainingtransfer were explored as this is the most relevant dimension for the training

intervention under investigation here.


5/22

Baldwin and Ford (1988) identified three categories of influences on training transfer;individual characteristics, training design factors and organisational factors. A reviewof literature has demonstrated that research exploring the relationship betweenorganisational, or the transfer system, factors and training transfer has beeninvestigated less frequently than training design or indivdual characteristics (Valada

et al. 2007; Saks & Belcourt, 2006; Chen, Holton & Bates 2006). The transfer systemrefers to all the person, training, and organisational related factors that have thepotential to influence transfer of learning to job performance (Holton, Bates andRouna, 2000). The complex events such as outcomes of a training interventioncannot be understood by analysing them in isolation because learned skills at theindividual level are embedded in a wider context.

Noe (2000) suggests that an understanding of organisational factors affectingtransfer will make a greater contribution to HRD practitioners wishing to optimise theeffectiveness of training and development programmes. Holton et al. (2000) statedthat an understanding of what constitutes an organizational transfer climate is

unclear and there is no clear consensus on the network of factors affecting transferof learning in the workplace. This is evident from the different explanations of thework environment. Lim and Johnson (2002) however, stated the work environmentfactors can be separated into two subcategories: factors that relate to the worksystem and people related factors. The following are the organisational transferfactors investigated in the current study:

3.1 Lim and Johnson (2002) found that the most important organisational systemfactor affecting transfer of training at the individual level is supervisor supportandinvolvement. However, Nijman, Nijhof, Wognum and Veldkamp (2006) argued

that empirical research does not unambiguously confirm this relationship andprovides contradictory results. We explore this factor in the current study.

3.2 Another level of support shown to influence transfer of training is peersupport. Nijman et al. (2006) described peer support as the extent to which peersbehaviour optimises trainees implementation of learning on the job. Peer supportaffects motivation to transfer (Noe 1986) and has been shown to predict theperceived opportunities to use learning (Quinones, Ford, Sego & Smith, 1995).The health care setting where this study is situated is characterised by staffshortages and increased workloads which may influence peer support.

3.3 An open communication climate was identified by Lim and Johnson (2002)as a work system factor found to influence training transfer. The manner in whichhealth care organization departments and units are organised can add to staffpressures (Clarke, 2007), which can affect the level of support for trainingtransfer received. It may influence trainees opportunity to perform or, it mayaffect communication between departments (Kupritz, 2002). Therefore, thevariable communication between departmentswas included in the study.

3.4 According to Holton et al. (1997), opportunity to use is influenced byorganizational factors (such as department goals and values), individual factors

(confidence to use new skills) and contextual factors (pace of work). It is alsoinfluenced by the provision of adequate resources and carrying out tasks that


6/22

enable use of new skills. Furthermore, the opportunity to practice skills onimmediate return to job can impact on skill retention.

3.5 Lack of adequate resources and failure to provide staff cover or reduce

workload while attending or implementing training can impact on motivation totransfer training. Gregoire (cited in Clarke 2002), identified a lack of time andresources and daily demands of the job as major work-related impediments to theuse of training. Therefore the variableresources were included in the study.

3.6 Huczynski and Lewis (cited in Clarke 2002), found work overload and crisiswork significantly impeded transfer. With regard to task constraints however,Clarke (2002) stated there have been mixed results in studies. However, thisstudy was conducted in the health care sector where work overload and crisiswork is a feature and where, according to Clarke (2007) there has been a lack of

transfer research. Consequently, this study included the organisational variable ofcompeting prioritiesas part of the investigation of factors affecting transfer.

3.7 The final organisational transfer system variable considered for its influenceon training transfer is trainee level of interestbecause, according to Elangovanand Karakowsky (1999) and Santos and Stuart (2003), training transfer will occurwhen trainees have both the ability and motivation to acquire and apply newskills.

In sum, the training evaluation and effectiveness literature indicates that HRDresearchers should engage in more comprehensive studies of the organisationalfactors that can affect training transfer to further our understanding of trainingeffectiveness. This study explores how seven work-related and person-relatedorganisational transfer system factors impact on training transfer. The study alsoexplores how training transfer impacts on a number of employee level (jobsatisfaction, conference, efficiency) and organisation-level outcomes (perception ofresident outcomes and quality of service delivered). Figure 1 presents an organisingframework for the study.


7/22

Figure 1:Organising Framework for the Study

4.0 Methodology

4.1 Method of inquiryA cross sectional survey was used to gather data from a range of subjects withdifferent demographic and situational characteristics. A mixed methods ortriangulation approach was also utilised. Both qualitative and quantitative methodsof inquiry were utilised which involved the use of questionnaires with open and

closed questions and semi structured interviews. Finally, data was gathered fromdual perspectives; participants who took part in the intervention and theirmanagement.

4.2 Training intervention under investigation

The intervention utilised for the purposes of the study was a 60 hour trainingprogramme designed specifically for the heath sector and the subsector (elder care)being investigated. The Activity in Care Training (ACT) programme provides courseparticipants with the knowledge and skills to develop chair-based physical activityprogrammes appropriate to needs of residents in long stay facilities for older adults,

the setting in which this investigation takes place.

Organisationtransfer systemWork related

(Collective) Level

of interest

Opportunity to use

Competing

priorities

Communication

between

departments

Resources

(management

only)

Person related

Peer support

Supervisor

support

Transfer of training

Delivering

Regular delivery

Outcomes

Employee

Job Satisfaction

Confidence

Efficiency

Organisation

Perceptions ofresidentoutcomes

Services offered

Quality

Satisfaction

Employee value


8/22

4.3 Sample and Procedure

The sample groups used for this study were the 204 public sector employees whocompleted the training intervention across 15 courses and 102 facilities plus their line

managers. Questionnaires containing both quantitative and qualitative items weremailed to the 204 participants/employees some time after completing their trainingprogramme. A total of 124 usable surveys were returned representing a responserate of 61%.

A non-probability approach for the selection of management respondents wasadopted; management personnel were identified by the participating employees toensure the appropriate person was targeted for the study. The management samplepopulation was 103. A total of 43 usable questionnaires were returned frommanagement respondents representing 42% of the sample population.

Interviews were conducted using a non-random, convenience sample with tenemployees and five managers.

4.4 Measures

4.4.1 Training outcomesThe training outcome variables investigated were grouped into employee andorganisational categories. In the healthcare industry, the outcomes of this trainingintervention are expected to be largely intangible or difficult to quantify. Thus, openended or part open ended questions were provided to seek further clarification onintangible outcomes.

Outcomes - Employee

Job satisfaction. Employee job satisfaction was measured using a three-scale item.Items included personal satisfaction with your job and attitude to the organisation.The Cronbachs alpha co-efficient for the three-item scale was .77. Managementperceptions were also measured using a three item scale with similar items and theCronbachs alpha co-efficient was .85.

Confidence. Employee confidence was measured using a three-scale item. Itemsincluded confidence to do physical activities with residents and confidence inperforming other work related activities with residents. The Cronbachs alpha co-efficient was .87. Management perceptions were measured using a four-item scaleand the Cronbachs alpha co-efficient was .90.

Efficiency. Employee efficiency was measured using a four-scale item includingattention to safety performing other duties with residents and ability to respond toresidents needs. The Cronbachs alpha co-efficient was .87. Managementperceptions were measured using a five item scale including level of supervision

required to do activity sessions. The Cronbachs alpha co-efficient was .80.


9/22

OutcomesOrganisation

Benefits to Residents: Employee perceptions of benefits to residents as aconsequence of participating in the activity sessions were measured by two nominal

items and four sets of scales representing a total of 14 items. The four scalecategories comprised of social benefits, cognitive benefits, psychological benefitsand physical benefits. The Cronbachs alpha co-efficient for the four combinedscales was .84. Measurement items were generally similar for management withsome modifications. Each of the four scales had alpha scores above .8.

Quality of service offered. Quality of services offered was measured by a six-itemscale. Sample items include changes in quality of services offered, number ofresidents attending the session and time allocated to activities overall. TheCronbachs alpha co-efficient was .92. The management perspective was measuredby three nominal items presented in closed question format such as improvements in

the quality of the chair based activity session offered and the number of residentscatered for.

Satisfaction with quality of service. Satisfaction levels with quality of services offeredwere measured by a three-item scale including resident satisfaction with theactivities offered and satisfaction of residents relatives with the overall activityprogrammes offered. The Cronbachs alpha co-efficient was .88. Managementsatisfaction levels were measured using one nominal item looking at residentsatisfaction.

4.4.2 Organisation factors affecting transfer

Level of interest. Level of interest was measured using a two-item scale: I enjoydelivering the session andthe residents enjoy taking part in the sessions.TheCronbachs alpha co-efficient was .69. The management perspective was measuredusing a three-item scale including the motivation of the relevant staff members, andmanagement policy to offer regular physical activity sessions.The Cronbachsalpha co-efficient was .79.

Opportunity to use. Opportunity to use was measured using a two-item scaleincluding delivering activity sessions is included in my job specification andthe

activity sessions are allocated a specific time. The Cronbachs alpha co-efficientwas.72. The management perspective was measured using a two-item scale. TheCronbachs alpha co-efficient was .86.

Competing priorities. Competing priorities was measured using a four-item scaleincluding it is too busy on my ward/unit to have regular sessions,and time isalways made on my unit/ward for the activity sessions. The Cronbachs alpha co-efficient was .78. The management perspective was measured using a three-itemscale including time remaining after taking care of the nursing and caring duties.The Cronbachs alpha co-efficient was .80.

Supervisor Support. Supervisor support was measured using a three-item scaleincluding I developed a plan with my line manager to set up the activity sessions,


10/22

andmy line manager is not aware of how the activities are going.The Cronbachsalpha co-efficient was .73. The management perspective was measured using afour-item scale including personally setting time aside to discuss the progress of thesessions with the ACT leader. The Cronbachs alpha co-efficient was .79.

Peer support. Peer support was measured using a three-item scale including theother staff encourage me to do the activities. The Cronbachs alpha co-efficient was.75. The management perspective was measured using a two-item scale includingthe co-operation of other staff getting residents ready for the sessions. TheCronbachs alpha co-efficient was .89.

Communication between departments. Communication was measured using a two-item scale including it can be difficult to get the information I need about residentsfrom other units.The Cronbachs alpha co-efficient was .72. Due to the subjectivityand experiential nature of this construct, management responses were not sought.

ResourcesThe resources variable was measured using a three-item scale including staffshortages make it difficult to do the sessions on a regular basis. However, as theCronbachs alpha co-efficient was .26, this variable was eliminated from theemployee analysis due to its lack of reliability. The management perspective wasmeasured using a four-item scale including the ability of the ACT leaders to set upthe session without assistanceand the availability of a dedicated room for theactivities. The Cronbachs alpha co-efficient was .82.

3.4.3 Transfer of Training - Direction

Delivery training program learningThis variable sought to establish baseline data with regard to the implementation oftraining within respondents organizations. Employee responses were measured byfive items at a nominal level which included questions such as are you currentlydelivering chair based physical activity sessions within your organization .Management responses were measured using a two item scale such as to yourknowledge, are the (ACT leaders) currently leading chair based physical activitysessions within your organisation.

Delivering regularlyRegular delivery was defined as delivering a minimum of one session per week. Thisvariable was established by the researcher from the recoding of responses relatingto the item on the frequency with which the chair based physical activities areimplemented.

4.5 Data AnalysisThe quantitative data enabled the production of descriptive statistics and qualitativedata provided explanations of constructs, actions and contexts observed by therespondents. Qualitative data was gathered through the open ended questions on

the questionnaires and the interviews. In both instances, respondents commentswere reviewed and content analysed prior to grouping them into specific categories


11/22

or themes according to what Scanlan et al. (cited in Biddle, Markland, Gilbourne,Chatzisarantis and Sparkes (2001) refer to as a deductive approach to contentanalysis. However, this was not feasible in all cases and therefore, any themes orquotes that did not fit into these categories were organised into new categories.This data was used to support the quantitative data.

5.0 Findings

Employee respondents ages ranged between 27 and 69 years. The mean age was49 (S.D. 7.9). Length of employment ranged from 3 to 37 years with the meannumber of years 13.1 (S.D. 7.6). Management respondents were responsible forbetween one and eight employees who had completed the intervention. The meannumber of employees for whom respondents were accountable was 2.3 (S.D. 1.3)with 11.4% responsible for four or more employees. The mean length of time theyhave been line manager to the trained personnel varied from 6.9 years for the first

employee to 5.4 years for the fourth employee. The average number of olderresidents per organization ranged from eight to 420 residents with the mean number58 (S.D. 64.6).

5.1 Transfer of Training

5.1.1 Delivery of Chair Based Physical Activity Sessions (training program content)& Regular Delivery

Sixty eight percent of employee respondents were delivering chair based physical

activity sessions at the time of completing the questionnaire. Forty eight per cent (n.84) of employee respondents were delivering the sessions on a regular basis (atleast once per week), while 26% did not indicate the frequency of delivery. The meannumber of sessions delivered per week was 2.37 ( S.D. 1.42).

Of the 32% employees who are no longer delivering sessions, 50% either neverimplemented them or stopped shortly after completing the intervention and 31% donot expect to resume delivering sessions in the future. Reasons cited for notimplementing the sessions were: inability to release staff from other duties (46%),lack of employee interest (17%), employee appointed to a new role (14%), theactivities being delivered by other staff members (11%) and change in employee

circumstances (11%).

These results provide critical evidence that training has been transferred at somelevel.

5.2 Impact of the training and development programme

5.2.1 Employee outcomes

The mean scores for perceptions of the employee outcomes on job satisfaction,confidence and efficiency from both employees and management perspectives are

presented in Table 1 below using the likert scale where 1 represents stronglydisagree and 5 represents strongly agree. These results indicate that both sets of


12/22

respondents agree employees work-related self efficacy and work efficiency hasimproved as well as their job satisfaction. Furthermore, whilst other factors mayhave contributed to these changes, the results also show respondents agree thatthey are also linked to participation in the training intervention. These results arepresented in table 1 below.

Table 1: Mean scores for perceptions of intervention impact on employees attitudeand behaviour

Dependent Variable Employee Management

n. Mean S.D n. Mean S.D

Job Satisfaction 102 4.2 .86 42 4.0 .67

Confidence 107 4.3 .86 42 4.2 .73

Efficiency 103 4.4 .79 40 4.1 .72

Extent to which the

intervention wasresponsible

105 3.7 .88 43 3.4 .90

1 = Not responsible 5 = fully responsible

5.2.2 Organisational OutcomesNinety seven percent of employees believe some residents have benefitted to somedegree as a consequence of taking part in the activity sessions. The mean score forthe combined scales of changes is presented in table 2 below with 1 representingalmost never notice these changes and 5 representing almost always notice.Management results supported employee perceptions.

The mean scores for employee perceptions of the degree of positive change inrelation to the quality of services offered, satisfaction with the quality of serviceoffered and perceptions of employee added value to the organisation or team arepresented in Table 2 below with 1 representing no change and 5 representing acomplete change. Management results generally indicated support for employeeperceptions with 98% believing there was an improvement in the quality ofprogrammes offered, 90% indicating there was an improvement in residentsatisfaction with the sessions provided and 78% believing there was an improvementin the number of residents attending the programme. Table 2 provides a summary ofthese results.

Table 2: Organisational outcomes summary of means and frequencies

VariableEmployee Management


Improvement in residents -combined scales

79 3.7 .61 38 3.5 .7

Quality of services offered scale 60 3.5 1.02

Satisfaction with quality of servicesoffered scale

99 3.1 1.14

Staff versatility/value scale 105 4.3 .87


13/22

N % Yes %No

Don

tknow

Improvement in quality of chair

based physical activity programmesoffered

41 90. 7.3 2.4

Improvement in number of CBPAoffered

40 75 22.5

2.5

Improvement in number ofresidents attending

40 77.5 20 2.5

Improvement in residentssatisfaction with the CBPA

38 89.5 5.3 5.3

Improved staff teamwork 39 76.9 15.4

7.7

Qualitative Data

The employee questionnaire had an open ended section inviting respondents toelaborate on any positive changes or personal benefits experienced which theybelieved was a consequence of participating in the intervention. The top six ratedexperiences presented in table 3 below are ranked according to the frequency ofcitation. Whilst the emerging topics or items largely reflect the constructs exploredwithin the study, the reference to a recognised qualification presented a new themeto the findings.

When all comments were collated into themes, job satisfaction was the only personaloutcome cited. However, the comments provided more insight into why and howrespondents perceive they have benefited personally from attending the intervention.For example, job satisfaction had improved because they have developed a betterrelationship with the residents as a result of providing activity sessions and becauseit highlights the social aspect of our care. One respondent indicated that it mademe stay in the older adult ward. Reference to increased awareness was made byseveral respondents such as knowing more (now) about illness and frailty andbeing more aware of health and safety issueswith residents.

Regarding organisational changes, employees were asked to explain what theybelieved was the most important change to have taken place within theirorganisation or unit as a result of the intervention. The most frequently citedoutcomes were resident-related benefits. For example, being more open andexpressing themselves or interacting more with other residents and staff.Interestingly, a change to the attitudes and support from other staff members wasthe second most frequently cited category of outcomes by respondents. Forexample, respondents referred to an increase in staff awareness of residentscapabilities increased and management recognition of the improvement in quality oflife through activities (increased).


14/22

Specific employee qualitative data shed greater light on the significance of thosechanges for the organisation. For example, before the course, we never had anactivity programme on long stay wards and, we now have protected time allocatedin order to perform activities as well as we now have a dedicated physical activityteam with a good skill mix. In terms of changing the approach to resident care,

employees indicated there is a new way of living opened up to residents. From amanagement perspective, comments such as (the training programme) hasbroadened out into other activities for residents and there is more (emphasis) onpsychological care or, that trained staff are perceived to be spending increasedquality time with residents and showing more in-depth understanding of patients asindividuals reflect an increasing focus on health promoting and quality of lifeservices for residents which constitute level 4 results

In relation to the most important organisational changes to take place as aconsequence of the training intervention, qualitative comments provided informationwith regard to how services for older adults had improved. There is now a dedicated

team and not just one person providing the activities while another indicated thatactivities are becoming more important to our clients overall healthcare.A greatervariety and number of activities offered were mentioned, while another stated theactivity programmes offered were more appropriate because they take into accountthe mobility and mental well beingof the resident.

Table 3: Summary of qualitative data on training intervention outcomes employee/participant perspectiveQuestion Area of change Number of

times cited

Positive changes experiencedas a consequence ofparticipating in the intervention

Awareness or insight into theimportance of PA for older adults 11

Increased confidence to do theactivities and to do the job

8

Greater understanding ofresidents needs in terms ofactivities

7

Improved job satisfaction 4

Better skills to do the job 4

More valued member of the team

Have a recognised qualification

Applying new learning to othergroups

2

22

Personal rewards or benefits asa consequence of participatingin the intervention

Job satisfaction19

What is the most importantchange to take place in theorganisation or unit as a result ofthe training intervention?

Resident outcomes 26

Staff attitudes and input 14

Management attitudes and action 11

Programmes offered 10


15/22

Table 4 below provides a summary of managements perceptions with regard topersonal benefits received by employees.

Table 4: Summary of qualitative comments on training intervention outcomes-

management perspectiveQuestion Area of change Number of

times cited

Positive changesobserved in employees

Confidence in ability to do activities 6

Enhanced skills and learning 4

Improved relationship with residents 3

Other outcomes Employee value to the organisation 5

Organisational benefits (regardingservices/programmes offered)

5

Benefits to residents 3Personal rewards orbenefits to staffmember(s)

Confidence 8

Job Satisfaction6

Interviews with employees cited increased confidence as the most importantpersonal outcome followed by enjoyment (job satisfaction). Two respondentsdescribed how learning had been applied to different contexts beyond the workplace.An extension in the number and range of activities offered together with

improvements in the quality of the activities and the numbers of residents able toaccess the activities were also cited.

Feedback from the interviews with management concurred with the employeecomments. Regarding improvements in confidence, indicators included increasedself esteem and assertiveness and they work more autonomously as members ofthe hospital recreation group.

5.3 The extent to which the organisation transfer system influences transfer oftraining in public sector hospitals.

Only 51% of management respondents were satisfied with the level of activitysessions offered despite indicating almost 87% of trained employees are providingthem for residents at some level.

The employee and management mean scores for the extent to which they perceiveeach of the seven variables influence training transfer are presented in table 5below. In this instance, 1 represents strongly disagree, 3 represents neither agreenor disagree and 5 represents strongly agree (see Table 5)

Table 5: Mean scores for perceptions of the effect of organisation transfer system

variables on training transfer.


16/22

Variable Employees Management


Level of Interest Scale 108 4.6 .64 41 4. .736

Opportunity to use Scale 106 3.6 1.44 41 4. .952

Line manager support scale 99 3.5 1.19 41 3.7 .763

Peer support Scale 104 3.2 1.15 41 3.4 1.21

Competing priority Scale 92 3.1 1.21 38 3.6 .962

Communication Scale 97 2.5 1.27

Resources Scale 40 3.4 .950

According to the mean scores, both management and employees concur with regardto the top three influencers on training transfer from the seven organisation transfersystem variables investigated. The level of interest variable is perceived to be thegreatest influence on the extent to which the chair based physical activity sessions

are conducted followed by opportunity to use, support of line managerrespectively. The following three variables peer support, competing priorities andcommunication between departments were ranked in a different order by both setsof respondents as illustrated in table 5 above.

5.3.1 Qualitative Data on Organisational Transfer Factors

Employee Data

The employee questionnaire contained an open ended item asking respondents to

indicate what they believe had been the biggest influence for the extent to which theywere transferring training. Table 6 below summarises the comment themes in rankorder. In terms of positive influencers, the top four factors were manager or linemanager support, resident interest, support from other staff and allocation ofdedicated time. For example, in terms of line manager and peer support,respondents cited the backing of line managers or the fact that managers specificallywanting to see the activities implemented as important influencers. Other positiveinfluencers included the assistance of peers in getting residents ready for theactivities, or being allocated dedicated time to carry out the activities and having adefined role as an activity person within the organisation. With regard to detractorsfrom transferring training, staff shortages, insufficient time, resident (health or

cognitive) status and lack of support from other staff were the four top cited factors.Qualitative feedback also indicated that employees working in some departmentsmay be allocated to other activities rendering them unable to do the group activities.

While staff shortages, staff support, time and completing other duties feature highlyas disablers to transferring training, resident status is cited as second highest factor.For example, one respondent indicated that the residents dependency hasincreased. For this respondent, it had implications for being released from work orfor getting cover to do the activities because of the increased workload on staff dueto the higher dependency of residents. For other respondents, it was felt that thechange in resident status made them unsuitable for the activities.


17/22

Table 6: Summary of qualitative comments on organisational system factorsinfluencing training transfer employee perspective

Rank PositiveInfluencers

Numberof

citations

Rank Negative influencers Numberof

citations1 Management or

line managersupport

12 1 Staff shortages 12

2 Resident Interest 11 Time 12

3 Staff support 10 3 Residents (health orcognitive)status

9

4 Dedicated time 9 4 Staff support 7

5 Part of job 8 5 Assigned other duties 5

6 Personal interest 6 6 Where working 3

7 (Unit) whereworking

4 7 Changed job role 2

(Attending) thetraining intervention

4 Lack of Resources 2

9 Current positionand level ofautonomy

3 8 Other 4

Being part of anactivity team

3

Management data

With regard to the positive influencers, management respondents felt the motivationand dedication of the trained employees exceeded all other factors whereas staffshortages was the most frequently cited negative influence. Specific activities whichmanagement believe positively impacted on transfer include assigning protectedtime (to do the activities), including the activities in (their) job description andhaving a committed , dedicated activity person in place to undertake the activities.It is interesting to note that the characteristics of the trained personnel are both thehighest ranked enabler and third ranked negative influence according tomanagement comments. For example, one respondent indicated staff say they

have no time but we have dedicated time twice weekly for activities whilst anotherindicated that on occasions, I have to prompt the leaders to do the sessions. Table7 below presents a summary of management comment themes in rank order.

Table 7: Summary of qualitative comments on organisational system factorsinfluencing training transfer management perspective

Rank PositiveInfluencers

Numberofcitations

Rank Negative influencers Numberofcitations

1 Motivation anddedication of trainedstaff

9 1 Staff shortages 6


18/22

2 Staff support 3 2 Residents (health orcognitive) status

3

Client interest 3 3 Leader interest 2

Dedicated time 3 4 Too busy 1

Part of job 3 5 Staff teamwork 1

Managementsupport

3 6 Not enough peopletrained

1

6. Discussion and Conclusions

This study adopted a mixed method approach to measure the impact of a trainingand development intervention at individual and organisational level across 77 longstay hospital settings within the Irish health care sector. It also investigated theeffect of organisational factors on training transfer post training.

With regards to measuring training outcomes, the primary concern relates more todemonstrating a link between the training intervention and results observed,especially in the case of intangible outcomes which, according to Spitzer (2005),most evaluation models fail to accommodate. This was particularly relevant for thisstudy where outcomes were largely intangible and value measured in terms of cost-effectiveness would not be easily demonstrated. Results demonstrated a highpercentage of respondents had experienced intangible personal rewards such asimproved job satisfaction. Furthermore, qualitative data reports that employees

demonstrated how perceptions of improvements in their efficiency and confidenceare linked to their perceived job satisfaction and secondly, how they were motivatedby satisfaction in their work rather than extrinsic rewards.

With regards to the organisational outcomes explored for in this study, allrespondents agreed that almost all residents had experienced positive benefits fromparticipating in the activities delivered as a consequence of the training intervention.What is evident from these comments is that while training has impacted on oneaspect of patient care, it has done so at multiple levels; health benefits, patientaccess, quality control, dedicated time for activities and the expansion of arecreational and therapeutic programme. However, it was both the dual perspective

and qualitative methodology that detected these outcomes.

Qualitative feedback from both management and employees rank staff shortages asthe top inhibitor to transferring training which is not surprising in a sectorexperiencing staff cutbacks and moratoriums on staff recruitment.

5.2 Implications for Practice & Research

The use of qualitative research methods enabled a deeper understanding of theintangible outcomes of the training intervention at employee and organisational level

which was particularly important in a sector where training outcomes were expectedto be predominantly intangible. This has implications for the design of research


19/22

instruments, for the expertise required to design and administer such instrumentsand the expertise for analysing and interpreting the results. The use of qualitativeresearch methods have created a greater understanding of the organisationaltransfer system factors perceived by respondents to contribute to the extent oftraining transfer. It enabled the distinction between how the variables investigated

acted as both enhancers and inhibitors which quantitative methods did not do. Thequalitative feedback also indicated that different elements within some of thevariables were perceived to have different levels of impact on training transfer bymanagement and employees. This provides further support for the use of dualperspectives and triangulation of methods in evaluation research

5.3 Limitations of the study

Although a cross sectional approach was used to investigate 77 hospitals within theIrish healthcare sector, health care systems are very complex with powerful

subcultures. While the findings of this study contributed to our understanding of thetransfer process in this sector, it cannot be assumed that they will generalise to allsettings and sectors. Nonetheless, the integrative model and processes used tooperationalise the study can be adapted for use in other sectors. Another limitationof the study was the small number of management respondents. To make the studyoperational, the number of organisation system constructs investigated for effect ontraining transfer was limited to seven which did not capture all the potentialinfluencing variables on training transfer.

References

Alliger, G. M., Tannenbaum, S. I., Bennett Jr, W., Traver, H.and Shotland, A. 1997.A meta analysis of the relations among training criteria. Personnel Psychology. 50(2), pp. 341-358.

Alvarez, K., Salas, E. and Garofano, C., 2004. An integrated model of trainingevaluation and effectiveness, Human Resource Development Review, 3(4), pp. 385-416.

Baldwin, T. T. and Ford, J. K. 1988. Transfer of training: a review and directions forfuture research. Personnel Psychology, 41, pp. 63-105.

Broad, M. L. 2005. Beyond Transfer of Training. Engaging Systems to ImprovePerformance, USA: Pfeiffer.

Burke, L. A. and Hutchins, H. M. 2008. A study of best practices in training transferand proposed model of transfer. Human Resource Development Quarterly, 19 (2),pp. 107-128.

Cannon-Bowers, J. A., Salas, E., Tannenbaum, S.I. and Mathieu, J. E. 1995. Towardtheoretically based principles of training effectiveness: A model and initial empirical

investigation. Military Psychology,(7), pp. 141-164.


20/22

Chen, G. and Klimoski, R. J. 2007. Training and development of human resources atwork: is the state of our science strong? Human Resource Management Review,17(2), pp.180-190.

Chen, Hsin-Chih, Holton lll, E. and Bates, R. A. 2006. Situational and demographic

influences on transfer system characteristics in organisations. Performanceimprovement quarterly, 19 (3), pp. 726.

Cheng, E. W. L., and Ho, D.C. K. 2001. A review of transfer of training studies in thepast decade. Personnel Review,30 (1), pp. 102-118.

Clarke, N. 2002. Job/work environment factors influencing training transfer within ahuman service agency: some indicative support for Baldwin and Fords transferclimate construct. International Journal of Training and Development, 6(3) pp. 146-162.

Clarke, N. 2007. The limitations of in-service training in social services. In: S.Sambrook, & J. Stewart (Eds.), Human resource development in the public sector.The case of health and social care). London: Routledge, pp. 315-340

Elangovan, A. R.and Karakowsky, L. 1999. The role of trainee and environmentalfactors in transfer of training: an exploratory framework. Leadership andOrganisation Development Journal, 20, (5), pp.268-276.

Ford, J. K. and Kraiger, K. 1995. The application of cognitive constructs to theinstructional systems model of training: implications for needs assessment, design

and transfer. In: C. L. Cooper, and I. T. Robertson,International Review ofIndustrial and Organisational Psychology. Chichester: Wiley, pp.1-48

Greene, J. and Caracelli, V. (eds.) 1997. Advances in Mixed-Method Evaluation:The Challenges and Benefits of Integrating Diverse Paradigms. Jossey-Bass, SanFrancisco.

Holton III, E. F. 1996. The flawed four level evaluation model. Human ResourceDevelopment Quarterly 7, pp. 5-21.

Holton lll, E. F., Bates, R. A., Seyler, D. L. and Carvalho, M.B.1997. Towardconstruct validation of a transfer climate instrument. Human Resource DevelopmentQuarterly, 8 (2), pp. 95-113.

Holton lll, , E. F., Bates, R. A. and Ruona, W.E.A. 2000. Development of ageneralized learning transfer system inventory. Human Resource DevelopmentQuarterly,11 (4), pp. 336-360.

Holton III, E. F., Chen, Hsin-Chih and Naquin, S. S. 2003. An examination oflearning transfer system characteristics across organisational settings. HumanResources Development Quarterly, 14 (4), pp. 459.


21/22

Hutchins, H. M. (2009). In the trainers voice: a study of training transfer practices. Performance Improvement Quarterly, 22(1), 69-93.

Hutchins, H. M. and Burke, L. A. 2007. Identifying trainers knowledge of training

transfer research findings closing the gap between research and practice.International Journal of Training and Development, 11 (4), pp. 236-264.

Kearns, P. 2005. From return on investment to added value evaluation: thefoundation for organisational learning.Advances in Developing Human Resources, 7(1), pp. 135-146.

Kupritz, V. W. 2002. The relative impact of workplace design on training transferHuman Resource Development Quarterly, 13 (4), pp. 427-448.

Lim, D. H. and Johnson, S. D. 2002. Trainee perceptions of factors that influencelearning transfer. International Journal of Training and Development, 6 (1), pp. 36-48.

Martin, H. J. 2010. Workplace climate and peer support as determinants of trainingtransfer. Human Resource Development Quarterly , 21 (1), pp. 87-104.

Mayo, A 2000. The role of employee development in the growth of intellectual

capital. Personnel Review,29 (4) pp. 521-533.

Nijman, Derk-Jan J. M., Nijhof, W. J., Wognum, A.A. M. and Veldkamp, B. P 2006.Exploring differential effects of supervisor support on transfer of training.Journal ofEuropean Industrial Training, 30 ( 7) pp. 529-549.

Noe, R. A. 1986. Trainees attributes and attitudes: neglected influences on trainingeffectiveness. Academy of Management Review , 11 (4), pp.736-749.

Noe, R. A. 2000. Invited reaction: development of a generalized learning transfersystem inventory. Human Resource Development Quarterly, 11 (4), 361-366.

Onwuegbuzie, A. J., Burke Johnson, R., and Collins, K. M. T. 2009. Call for MixedAnalysis: A Philosphical Framework for Combining Qualitative and QuantitativeApproaches. nternational Journal of Multiple Research Approaches. 3 (2), pp.114-140.

Quinones, M. A., Ford, J.K., Sego, D.J. and Smith, E.M. 1995. The effects ofindividual and transfer environment characteristics on the opportunity to performtrained tasks. Training Research Journal,1, pp. 29-48.

Rouiller, J. Z. and Goldstein, I. L. 1993. The relationship between organisational

transfer climate and positive transfer of training. Human Resource DevelopmentQuarterly.4 (4), pp. 377-399.


22/22

Ruona, W.E.A., Leimbach, M, Holton lll, E.F. and Bates, R. A. 2002. The relationshipbetween learner utility reactions and predicted learning transfer among trainee.International Journal of Training and Development, 6(4), pp.218-228.

Saks, A. M. and Belcourt, M. 2006. An investigation of training activities and transferof training in organisations. Human Resource Management, 45 (4), pp. 629-648.

Salas, E. and Cannon-Bowers, J. A. 2001. The science of training: adecade of progress.Annual Review of Psychology, 52, pp. 471-499.

Santos, A. and Stuart, M. 2003. Employee perceptions and their influence ontraining effectiveness.Human Resource Management,13 (1), pp. 27-45.

Spitzer, D. R. (2005). Learning effectiveness measurement: a new approach formeasuring and managing learning to achieve business results. Advances in

Developing Human Resources, 7(1), pp. 55-71.

Tan, J., Hall, R. J. and Boyce, C.(2003) The role of employee reactions inpredicting training effectiveness,Human Resource Development Quarterly Vol. 14:,No. 4. Pp. 283-296.

Tannenbaum, S. I. and Woods, S. B. 1992. Determining a strategy for evaluatingtraining: operating within organisational constraints. UHR. Human ResourcePlanning, 15 (2), pp.63-82.

Tashakkori, A. and Teddlie, C. (Eds.) 2003. Handbook of Mixed Methods in Social &Behavioral Research,Sage, Thousand Oaks.

Van der Klink, M. R., Gielen, E. E. M., and Nauta, C. 2001. Supervisory support as amajor condition to enhance transfer. International Journal of Training andDevelopment, 5 (1), pp. 52-63.

Wang, G. G. and Spitzer, D. R. (2005). Human resource development measurementand evaluation: looking back and moving forward.Advances in Developing HumanResources. 7 (1), pp. 5-16.

Wang, G. and Sun, J. Y. (2009). Perspectives on theory. Clarifying the boundaries ofhuman resource development. Human Resource Development International, 2 (1),pp.93-102.

training effectiveness and transfer -- a mixed methods investigation

Documents