eprints.whiterose.ac.ukeprints.whiterose.ac.uk/147062/1/manuscript_revised... · web viewthe total...

Setting priorities for health education research: A mixed methods study in Australia

Abstract

Introduction: Identifying priority research topics that meet the needs of multiple stakeholders should maximise research investment.

Aim: To identify priorities for health education research.

Methods: A three-stage sequential mixed methods study was conducted. Priorities for health education research were identified through a qualitative survey with 104 students, patients, academics and clinicians across five health sciences and 12 professions (stage 1). These findings were analysed using framework analysis and transposed into a quantitative survey whereby 780 stakeholders rated and ranked the identified priorities. Descriptive statistics identified priorities, exploratory factor analysis grouped priorities and differences between stakeholders were determined using Mann-Whitney U tests (stage 2). Six individual or group interviews with 16 participants (stage 3) further explicated the results from previous stages.

Results: Of 30 priorities identified, the top were: how best to ensure students develop the required skills for work; how to promote resiliency and well-being in students; and ensuring the curriculum prepares students for work. For the majority of priorities, no significant differences were found between different stakeholder groups.

Conclusions: These findings will be used to inform health educational research strategy both locally and nationally. Further research should explore if setting priorities can be translated effectively into education research policy and practice.

Practice points

· Engaging in systematic and consultative processes with multiple stakeholders to identify health education research priorities is essential to maximise the efficiency and impact of health education research.

· The highest-priority research areas identified were: ensuring that students develop the required skills for work/practice; how to promote resiliency and wellbeing in students; and ensuring the curriculum prepares for work/practice.

· Research priority factors identified were: (i) Culture of learning together in the workplace; (ii) Preparation for work; (iii) Meeting future Australian healthcare needs; (iv) Pedagogical effectiveness; (v) Workforce issues; and (vi) Curriculum integration.

· These research areas were prioritised as they were perceived to improve student learning, were seen as a current educational problem, supported the development of evidence-based education, and improved the quality of care and patient safety and preparedness for work.

· The need still remains to determine if priority setting exercises can be translated effectively into education research policy (e.g. strategy) and practice (e.g. focused education research effort, enhanced education research quality).

Introduction

Setting priorities for any research is essential to maximise the impact of efforts (Dechartres and Ravaud 2015). Establishing research priorities focuses limited resources on investments that are relevant to the context, which should lead to greater impacts (Viergever et al. 2010) and minimise waste (Dechartres and Ravaud 2015). In the area of health (and disease specifically) this has the potential for greater improvements in population health and wellbeing. Based on these drivers, it is fair to say that in the field of health professions and sciences education, identifying and focusing research effort on priorities should improve our understanding of how best to prepare health professionals and scientists for work, which has the potential to improve health care delivery (Dechartres and Ravaud 2015).

Health education research priorities

A number of health professions education research priority setting studies have been conducted across the world over recent years (Wilkinson et al. 2010; Hodges et al. 2011; Tootoonchi et al. 2012; Dennis et al. 2014; Ajjawi et al. 2017). These studies have used a range of approaches, with the common thread of gathering input from a range of people with a key stake in health professions education, including patients, students, academics and clinicians. To date, these exercises have been conducted across medical and dental education research in Scotland, Canada and New Zealand. Common priorities identified across these studies include professionalism (Wilkinson et al. 2010; Ajjawi et al. 2017), assessment (Wilkinson et al. 2010; Dennis et al. 2014; Ajjawi et al. 2017), feedback (Wilkinson et al. 2010; Dennis et al. 2014), preparedness for practice (Hodges et al. 2011; Dennis et al. 2014; Ajjawi et al. 2017) and workforce issues (Wilkinson et al. 2010; Dennis et al. 2014; Ajjawi et al. 2017). These priority setting endeavours in health professions education have however been criticised for not engaging appropriate samples, being too focused on local contexts and being methodologically suboptimal (Worley and Schuwirth 2014). Nonetheless, they have been proposed to support the development of local research strategy, taking into consideration context to ensure meaningful and relevant research questions are asked and thus answered (Ajjawi et al. 2017). To the authors’ knowledge no health education research priority setting studies have been conducted in Australia and none of the previously completed studies have focused on priorities across multiple health professions and sciences.

Healthcare professionals play an important role in the delivery of healthcare, and thus understanding education research priorities for these groups (in addition to medicine) is fundamental to ensuring all health graduates are optimally prepared for work. Although there are no examples of education research priority setting studies in nursing or the allied health professions, several general (and profession-specific) research priority setting studies have identified education research topics as priorities. For example, Rankin et al. (2012) conducted a study using a modified Delphi technique to define the national research strategy for physiotherapy in the United Kingdom. Four expert panels reflecting the core areas of physiotherapy practice were established. Each panel consisted of between 40 and 61 members representing a range of stakeholders (e.g. clinicians, researchers, service users, management, policy-makers etc.). Data were collected over three online surveys (called rounds). Each round had a minimum of 53% response rate. The first round asked participants to list their research priorities and yielded 592 topics. The second and third rounds sought to rank and establish consensus around the topics suggested. While the vast majority of research priorities identified were clinical in nature (e.g. determining best practice for upper limb rehabilitation post-stroke), one priority identified by one of the panels was about the pre- and post-registration education of physiotherapists to ensure their capacity to support health behaviour change. Although this education research priority was identified, it was not elaborated on in the paper. Furthermore, while consensus of priorities was achieved, there was no exploration of the reasons underlying participants’ selection of priorities providing little indication of why this specific education priority was identified.

Monterosso et. al (2015) employed a modified two-round Delphi technique to develop a nursing research strategy in two large private tertiary hospitals in Australia (Monterosso et al. 2015). Of the 448 nurses and midwives invited, 86 (19.2%) agreed to participate in the expert panel for the first round (survey) in which participants were asked to list up to five problems or questions relating to nursing and family care of patients which should be researched. Of the 257 topics identified, 23 were considered potential research topics, with 10 of these focusing on learning and development. For the second round, only members of the Learning and Organisational Development Department were invited to participate. Ten of the 15 members responded to a survey which sought to rank and refine the learning and development research priorities. The majority of subtopics related to the continuing education and professional development of qualified nurses and midwives, with one subtopic specifically focusing on undergraduate nurse education (e.g. the effects of third-year undergraduate students mentoring first-year students). While the researchers attempted to engage all nurse and midwife employees within the study settings, there was no evidence of considering other stakeholders’ priorities (e.g. senior managers, service-users) (Monterosso et al. 2015). Other consensus-based studies have established priorities related to education research. For example Considine, Curtis, Shaban & Fry (2018) employed the Delphi Technique to establish research priorities in emergency nursing and found that research in the educational preparation of clinicians to be of high importance (Considine et al. 2018). There remains a need to explore priorities of other health care professionals.

Rationale behind health education research priorities and differences in perspectives

While identifying priorities is a key first step, understanding the reasons behind prioritising certain health education research topics assists with the development of shared research agendas. Previous health education research priority setting studies have identified patient safety, quality of healthcare, investing for the future workforce, policy agendas and evidence-based education as common reasons underpinning the various priorities (Dennis et al. 2014; Ajjawi et al. 2017). This information provides important context to explain the selection of priorities that can help in research strategy development. In addition, differences in perspectives on priorities assists interpretation and priority refining. In previous work, differences in identified medical and dental education priorities have been found in participants of different age, sex, ethnicity and role in health care (Dennis et al. 2014; Ajjawi et al. 2017). This emphasises the need to involve multiple different perspectives from those with varying roles in health education.

The need for this study

As mentioned above, to our knowledge no previous studies have set out to identify health education research priorities across multiple health science and professional groups. Therefore, the current study aimed to answer the following research questions across 12 healthcare professions and five health sciences in one Australian University with two large faculties with campuses in two countries with a view to inform the development of a health education research strategy: (i) what are the health education research priorities over the next 3-5 years according to multiple stakeholders?; (ii) what is the rationale provided by multiple stakeholders for prioritising specific health education research topics?; and (iii) what are the similarities and differences in health education research priorities across the range of stakeholders?

Methods

Design

A sequential mixed methods study based on previous priority setting methodology (Dennis et al. 2014; Ajjawi et al. 2017) was conducted (Figure 1). This involved the following stages: (stage 1) anonymous qualitative online survey to identify health education research priorities and the reasons behind these priorities; (stage 2) anonymous quantitative online survey to determine top priorities and the reasons why; and (stage 3) individual and group interviews to further explicate the results from previous stages. Ethics approval was obtained from [removed for blind peer review] and [removed for blind peer review] (approval numbers 7816 and 17-0000-326XL).

Setting

The study sample included a range of stakeholders in health education and health education research drawn from an Australian University with two large faculties with campuses in two countries: (1) Medicine, Nursing and Health Sciences; and (2) Pharmacy and Pharmaceutical Sciences. Together these faculties provide education for 12 different healthcare professions and five health sciences courses.

Sampling

A convenience sample of students, educators, researchers, clinicians and clients/patients across all healthcare professions and health sciences from the university, major teaching hospitals and the community (urban and rural) were invited to participate in each stage of the study. A variety of perspectives was sought to capture any differences in research priorities across healthcare professionals, health sciences, educators, researchers and service users.

Recruitment

For stage 1, key individuals were identified by the research team as having a strong or strategic interest in health education and/or health education research within the university and were invited to complete the qualitative survey. Participants from stage 1 were then invited to volunteer for stage 2, with this sample being substantially expanded to a broader group in order to maximise input into the quantitative survey. Finally, stage 3 interview participants were selected conveniently from stage 2 respondents who indicated willingness to discuss the study findings in more detail. For each stage, participants were recruited via email from faculty networks, academic leads, mail lists, e-notices on virtual learning environments (students only), snowballing techniques and face-to-face encounters. For stages 1 and 2, the initial email invitation was followed by a reminder invitation at three weeks and a second email reminder at six weeks. Recruitment continued until contributions from all stakeholder groups were obtained.

Data collection

Stage 1 data were collected via an online anonymous qualitative survey delivered through Qualtrics survey software (©2018) 1. The survey asked participants to respond to open-ended questions about their first, second and third priority for health education research and the reasons underlying their selected priorities. Participants were also asked several demographic and stakeholder-related questions. The priorities identified from the qualitative survey analysis were then used to inform the Stage 2 data collection method: a quantitative survey.

Stage 2 data were collected via an online anonymous quantitative survey, again using Qualtrics (©2018)1. Participants were asked to rate the importance of the 30 research priorities on a Likert-rating scale from one (not important) to five (very important). These 30 priorities were identified in Stage 1, where 24 priorities were used from previous priority setting exercises (Dennis et al. 2014; Ajjawi et al. 2017) and six new priorities were identified. From the list of 30 identified priorities, participants were asked to select their top three priorities, describe the reasons underlying their selections, and list any additional priorities (not already provided) via a comment box. Participants were then asked to answer several demographic and stakeholder-related questions.

Stage 3 involved presenting the analysis of the quantitative survey data to students, teachers, clinicians and patients through one individual and five group interviews. Participants were asked about their perspectives on the health education research priorities identified from stage 2. They were then asked to describe what the priorities should be according to their own understandings and the reasons underlying the importance of the priorities for them. Finally, the participants were invited to give feedback to help with the development of the university’s health education research strategy, plus the proposed development of research networks based on the exploratory factor analysis of stage 2 data (described below).

1 Questionnaires can be requested from the corresponding author.

Data analysis

The stage 1 qualitative survey data were imported into NVivo (version 12 QSR International ©2018) and analysed using a team-based framework analysis approach (Ritchie and Spencer 1994). Accordingly, both inductive and deductive techniques were employed. Firstly, six authors (CR, PC, SP, CW, MM, JM) inductively analysed the data and then discussed their independent analyses. Existing priority setting frameworks (Dennis et al. 2014; Ajjawi et al. 2017) were drawn on during discussions to help make sense of team members’ analyses and to develop the coding framework for the current study. The researchers felt that this hybrid inductive-deductive approach was optimal given: (1) our professionally diverse team with backgrounds in nursing, physiotherapy, dietetics, radiation therapy, social science and psychology, with some having little familiarity with previous priority setting exercises and coding frameworks, and others having extensive knowledge of previous frameworks; and (2) our desire to compare our study findings with previous priority setting exercises (Dennis et al. 2014; Ajjawi et al. 2017). These six authors aimed to identify themes in the current data pertaining to existing frameworks, but also to detect new themes. The team identified six new priorities not already described in previous literature, with three new reasons underlying the priorities within the existing coding framework (Dennis et al. 2014; Ajjawi et al. 2017). In addition, three priorities that had been previously identified were not found in the current data. However, as mentioned above, it was decided to include these priorities in our stage 2 questionnaire.

Stage 2 analysis was undertaken by multiple authors (CP, CR, TB, LO, BW). Complete Likert-rating scale data (n=780) pertaining to the 30 priorities from the stage 2 questionnaire were analysed using non-parametric descriptive statistics (i.e. median and inter-quartile range for each priority). Ranked sum scores for completed data were calculated from the 629 participants’ who completed the questions on the top three identified priorities as follows: where the participant had rated a topic as having first priority, it was given three points; topics rated as being second and third were given two and one points respectively. The scores given to each of the 30 priorities by all participants were then summated to identify scores and thus rankings of importance for each item. Note that two indicators of importance were employed because: (a) Likert-rating scale data would allow examination of how the priorities clustered together; and (b) ranked sum scores would better differentiate between the different priorities in the case of ceiling effects.

Using Classical Test Theory principles, exploratory factor analysis using principal components analysis with direct oblimin rotation was conducted on the 30 Likert-rating scales to categorise higher-order factors in the data. Based on previous studies (Dennis et al. 2014; Ajjawi et al. 2017), 0.35 was used as a cut off for the inclusion of a variable on a factor. While the scree plot showed some flattening off after 7, 8 and 9 factors, six factors had eigenvalues more than one, plus the six-factor model seemed most parsimonious and easy to interpret. Once the six-factor structure of the subscales was established, factor scores were calculated for each participant for each factor, whereby the sum of Likert-rating rankings for the priorities for each factor was totalled. The total factor scores depended on how many items loaded on each factor (F1=8 items, scores of 8–40; F2=6 items, scores of 6–30; F3=4 items, scores of 4–20; F4=5 items, scores of 5–25; F5=6 items, scores of 6–30 and F6=4 items, scores of 4–20). Internal consistency of the factors was measured using Cronbach’s α.

Mann-Whitney U tests were undertaken to explore any differences in factor scores between health professional and health science stakeholders, role in health professions education (e.g. educator, student, patient, researcher, etc.), gender and cultural background. For the continuous variable (age), a Spearman’s rho coefficient was used to explore associations. Statistical significance was set at p<0.05. Finally, the text responses describing any new priorities and the reasons behind these priorities were analysed by one author (OK) using the same framework developed and refined in Stage 1. This qualitative coding was verified by a second author (CP).

Data from the stage 3 interviews were transcribed verbatim and analysed via NVivo (version 12 QSR International ©2018) using the same framework developed in stages 1 and 2. One author (OK) coded all data and four other authors verified the coding (HH, JM, TB, CP). Qualitative data related to “why” participants reported the priorities from both stages 1 and 2 were analysed together with the stage 3 interview data in order to explore the reasons given for priorities and assist in informing the health education research strategy going forward.

Insert figure 1 about here

Results

All stages of the study were completed by stakeholder groups across 12 health professions and five health sciences representing the courses offered at the study institution (Table 1). Our participants were generally reflective of the health workforce in Australia being female dominated, especially given the inclusion of allied health and nursing in this study, known to be majority female (Australian Government Department of Health 2018). The research questions will be addressed here collectively drawing on data from stages 1-3.

Insert table 1 about here

(i) The health education research priorities for the next 3-5 years at one Australian University with two large faculties and campuses across two countries according to multiple stakeholders

A total of 104 qualitative surveys were returned from 246 respondents who opened the email invitation to participate (41% response rate). Eight major health education research priority themes were identified with priorities relating to: (1) students; (2) educators; (3) working with others in the workplace environment; (4) culture of workplace learning environments; (5) curriculum integration; (6) curricula content; (7) curriculum delivery; and (8) assessment and feedback. Under these themes, 27 educational research priorities were identified. In addition, three priorities from coding frameworks identified in previous studies (Dennis et al. 2014; Ajjawi et al. 2017) were added for testing in Stage 2 (Table 2).

Although the quantitative survey was opened by 1179 individuals, between 629 and 780 participants completed key elements of the survey (53-66% response rate). Analysis of the 30 Likert-rating scales (n=780) and rank order of priorities (n=629) revealed the top three priorities as being: (1) understanding how best to ensure students develop the required skills for work; (2) understanding how to promote resiliency and well-being in students; and (3) ensuring the curriculum is fit for purpose (Table 3).

Insert tables 2 & 3 about here

Factor analysis revealed a clustering of priorities within six factors (Table 4). These six factors were identified as: (i) Culture of learning together in the workplace; (ii) Preparation for work; (iii) Meeting future Australian healthcare needs; (iv) Pedagogical effectiveness; (v) Workforce issues; and (vi) Curriculum integration.


(ii) The rationale provided by multiple stakeholders for prioritising specific health education research topics

In stage 1, the most commonly reported reasons underpinning participants’ priority selection were (in decreasing order of frequency): improved student learning (n=164); the priority topic being identified as a current educational problem (n=102); the importance of developing evidence-based education (n=102); and for improvements in quality of care (n=59) and patient safety (n=51). Consistent with stage 1, stage 2 participants most frequently reported improved student learning as the reason underlying their research priorities (n=505). A new reason code was identified in this stage of data collection: graduate employability or preparedness for career. This was the second most frequently stated reason at stage 2 (n=380). Other frequently stated reasons at stage 2 included: the priority topic being identified as a current educational problem (n=198); for faculty safety and wellbeing (n=183) and for student safety and wellbeing (n=180). Patient safety (n=153), the development of evidence-based education (n=106), investing in the future (n=102) and quality of care (n=81) were also commonly stated reasons at stage 2 (Table 5). In stage 3, participants often flagged the increasing problem of students’ mental health issues, which may explain the high priority around how to promote resiliency and well-being in students.


(iii) Similarities and differences in health education research priorities across a range of stakeholders

There were no statistically significant differences found between factor scores and participants’ gender, age or cultural background. There were also no significant differences in factor scores between health sciences and health professional participants, apart from factor 3: “meeting future Australian healthcare needs”. Scores for health sciences participants (median=16, IQR=14-19) were higher than for health professional participants (median=16, IQR=13-18) (Z= -2.124, p=0.034) for this factor 3. Furthermore, there were no significant differences in factors scores between students and non-student participants, except for two of the factors. For factor 1: “culture of learning together in the workplace” scores for students (median=32, IQR=29-35) were lower than for other participants combined (i.e. non-students: median=34, IQR=31-37) (Z= -3.639, p<0.001). For factor 4: “pedagogical effectiveness” scores for students (median=20, IQR=18-22) were lower than for other participants combined (i.e. non-students) (median=21, IQR=19-23) (Z= -2.467, p=0.014).

Discussion

This study aimed to investigate health education research priorities over the next 3-5 years at one institution across two health-related faculties and countries. It also aimed to explore stakeholder rationale for selecting these priority topics, and any differences in the research priorities identified across different stakeholders.

Summary of key findings

Through a sequential mixed method design engaging various stakeholders, our study identified the top three priorities as: understanding how to ensure students develop the required skills for work, how to promote resiliency and wellbeing in students, and ensuring the curriculum prepares students for work. Understanding how to develop communication skills, ensuring assessments lead to fitness for practice and providing more useful feedback and the role of workplace-based learning as part of health-related curricula, were also identified as high priorities through Likert ratings but not necessarily through rank (ranked 4th, 6th, 12th and 13th respectively). The priorities were grouped under six factors: (i) Culture of learning together in the workplace; (ii) Preparation for work; (iii) Meeting future Australian healthcare needs; (iv) Pedagogical effectiveness; (v) Workforce issues; and (vi) Curriculum integration. Priorities were typically similar across different types of participants; we found only three significant differences in the perspectives of different stakeholder groups.

Priorities compared with previous studies

This study across 12 health professions and five health sciences identified similar health education research priorities to previous studies (Wilkinson et al. 2010; Hodges et al. 2011; Dennis et al. 2014; Ajjawi et al. 2017). While understanding how best to ensure students develop the required skills for practice was consistently identified as a priority across this study and others (Dennis et al. 2014; Ajjawi et al. 2017), promoting resiliency and wellbeing was only identified by our study and one other (Dennis et al. 2014). All studies identified ensuring curriculum prepares students for work as a priority (Dennis et al. 2014; Ajjawi et al. 2017). There was also a consistent theme across other priority setting studies that identified factors related to workplace learning (Wilkinson et al. 2010; Dennis et al. 2014), workforce recruitment and retention issues (Wilkinson et al. 2010; Dennis et al. 2014; Ajjawi et al. 2017) and curriculum integration (Dennis et al. 2014; Ajjawi et al. 2017). However, factors related to meeting Australian healthcare needs and pedagogical effectiveness (including issues such as cost-effectiveness) were uniquely identified in this study. Synthesising the findings of this current study and previous studies suggests that education research related to workplace learning, preparation of the workforce and curriculum integration seem to be priorities irrespective of time-periods, disciplines and countries.

Reasons for priorities compared with previous studies

The reasons for prioritising these topics in education research were similar to previous studies (Dennis et al. 2014; Ajjawi et al. 2017). However, this current study also identified a broader range of reasons than those identified previously, perhaps due to this study including a broader range of stakeholders. For example, in the current study, graduate employability was a key factor for ranking priorities highly. This may be due to the sample of health sciences participants included in the current study (15% of the total sample of stage 2) as the employability of graduates of health sciences (and others science degrees) has previously been shown to be low compared with graduates of professional courses such as nursing and teaching, where there are recognised skills shortages (Jackson 2014; Choate et al. 2016; Jackson and Collings 2018). This finding might also be explained by the large number of allied health perspectives obtained, where employability is a key area of concern for many such professions due to practitioner shortages in some areas (Australian Government 2017). This focus on employability may also be due to the perspectives of participants that health practitioners are not being prepared adequately for the work they need to do in practice (Frenk et al. 2010 ; Ricketts and Fraher 2013; Gorman 2015).

Faculty and student health and wellbeing were also identified as the reason behind the selection of priorities. This reason was unique to this study and may reflect the impact that mental health has on competency development for students in health programs. The comparatively high prevalence of poor mental health in higher education students is not a new phenomenon. It has become increasingly acceptable to talk about and, given the potential adverse outcomes associated with poor mental health, societal pressure is mounting to address such concerns (VanderLind 2017; Bruffaerts et al. 2018). In the medical school student population a higher prevalence of poor mental health when compared with the general population is well-established (Heiman et al. 2018). Furthermore, the rates of burnout, depression and even suicide are generally greater in medical practitioners than members of other professions (Slavin et al. 2014). Kernan et al. (2008) established a link between medical students’ self-reported experiences of poor mental health, stress or relationship difficulties and the threat these conditions perceivably pose to academic performance (Kernan et al. 2008). This is also the case in paramedicine (Varker et al. 2018). More recently, in their research study investigating the link between mental health problems and academic performance in 4921 college students, Bruffaerts et al. (2018) found that students living with mental health problems (as measured by the World Health Organisation’s World Mental Health Survey) objectively demonstrated lower levels of academic achievement and functioning compared with their mentally-well peers (Bruffaerts et al. 2018). This present study further highlights a recognition of the fundamental nature of student and faculty health and wellbeing when considering the various aspects of the preparation of healthcare and health science students and setting priorities for education research. The influence of student and faculty wellbeing on academic and clinical performance, requires further exploration.

Similarities and differences across stakeholder groups compared with previous studies

Unlike other studies (Dennis et al. 2014; Ajjawi et al. 2017), there were almost no significant differences in factor scores between participants across different characteristics. This might suggest a general consensus of opinion amongst students, educators and clinicians across multiple health profession and science backgrounds at our one institution, compared to the previous studies, which explored differences in larger groups of stakeholders across multiple institutions in one country. Furthermore, we found that few statistically significant differences between participant groups in the current study may have been due to the smaller sample sizes of our sub-groups compared to the two previous studies (Dennis et al. 2014; Ajjawi et al. 2017), thereby reflecting insufficient power to detect significant differences (Type II error). Differences in student perspectives compared with non-student perspectives in relation to two factors (i.e. learning together in the workplace and pedagogical effectiveness) might suggest that students have not yet experienced workplace learning sufficiently to fully understand its important role in supporting learning. They might also already value the effectiveness of the teaching they have experienced. Indeed, differences in student and teacher perspectives around learning have been reported elsewhere (Limniou and Smith 2010). That health sciences participants scored one factor (i.e. meeting future Australian healthcare needs) higher than health professional participants might reflect consumer expectations for social accountability.

Methodological strengths and challenges of the study

Our sample was diverse and inclusive, including reasonable numbers of non-academic stakeholders (e.g. students, clinicians, patients) unfamiliar with the literature, thereby strengthening the study findings. Although our sample size was reasonable for a study of this kind, it was smaller and more heterogeneous than previous studies (Dennis et al. 2014; Ajawi et al. 2017), meaning that we may have lacked sufficient power to detect significant differences between sub-groups. While our study was conducted at one University, our participants were relatively geographically dispersed across metropolitan and regional areas of one Australian state, including participants from an overseas campus of the University. The similar characteristics, including female dominance, of study participants to other medicine, nursing, allied health and health sciences facilities across Australia (Brewer and Jones 2013; Brewer et al. 2017) may mean that our findings are transferable to other areas, particularly in [removed for blind review], to inform local research activity. The sequential mixed methods approach to identifying priorities allowed for participants in Stage 1 to have genuine input into the list of priorities presented in Stage 2, meaning that the priorities ranked in stage 2 were participant rather than researcher-derived. Whether these findings actually reflect research gaps or participants' perceptions of these gaps remains unknown. Given the dominance of educators/clinicians in our study, it is likely that the priorities to reflect current issues related to teaching and learning practice. Evidence suggests that the health education research community undertake opportunist research and there are gaps in methodological diversity and use of theory (Albert et al. 2007; Webster et al. 2015). Our work that has identified perceived research priorities at least provides structure and the potential for strategy to focus future efforts rather than being opportunistic. There is a need to ensure that research focussing on these priorities meet quality standards such that they are transferable across institutions and disciplines.

Study implications and conclusions

These findings will inform the development of a research strategy to focus educational research effort in our setting. Creating a strategy and education research culture will be important to maximise research efforts into the future. The importance of a positive research culture has been identified previously (Ajjawi et al. 2018). The relevance of an environment in which research is valued and supported, where leadership and a clear research strategy are evident, resources (e.g. funding, time and expertise) are available and researchers are able to develop collaborative relationships, has been reported as key ingredients to establishing an effective research culture (Ajjawi et al. 2018). Such environments where positive collegial relationships are fostered and where value is placed on research and leadership is evident, should cultivate positive attitudes and commitment to research (Holligan et al. 2011). While there is a growing number of studies identifying priorities for health education research, to our knowledge, there are no studies that examine whether identified priorities have influenced further research efforts and in what ways. Future research is therefore needed to determine if this (and other) priority setting exercises do impact upon research strategies going forward and if so how, with what consequences? The need for competency-based education to better reflect the requirements of healthcare workforce is a constant challenge. Having education research evidence to support curriculum redesign to better prepare the future health workforce is part of the solution to achieving this.

References

Ajjawi R, Barton K, Dennis A, Rees C. 2017. Developing a national dental education research strategy: priorities, barriers and enablers. BMJ Open. 7(3):e013129.

Ajjawi R, Crampton P, Rees C. 2018. What really matters for successful research environments? A realist synthesis. Medical Education. 52(9):936-950.

Albert M, Hodges B, Regehr G. 2007. Research in Medical Education: Balancing Service and Science [journal article]. Advances in Health Sciences Education. 12(1):103-115.

Australian Government. 2017. Labour Market Research Health Professions, Australia, 2017-18. [online]: Australian Government, Department of Jobs and Small Business; [updated 2017; accessed 2018 2 November]. https://docs.jobs.gov.au/system/files/doc/other/aushealthprofessions_2.pdf.

Australian Government Department of Health. 2018. Health Workforce Data. Canberra, ACT: Commonwealth of Australia; [updated 1 August 2018; accessed 2019 2 April]. https://hwd.health.gov.au/datasets.html.

Brewer M, Flavell H, Jordon J. 2017. Interprofessional team-based placements: The importance of space, place, and facilitation. Journal of Interprofessional Care. 31(4):429-437.

Brewer M, Jones S. 2013. An interprofessional practice capability framework focusing on safe, high-quality, client-centred health service. Journal of Allied Health. 42(2):45E-49E.

Bruffaerts R, Mortier P, Kiekens G, Auerbach R, Cuijpers P, Demyttenaere K, Green J, Nock M, Kessler R. 2018. Mental health problems in college freshmen: Prevalence and academic functioning. Journal of Affective Disorders. 225:97-103.

Choate J, Green J, Cran S, Macaulay J, Etheve M. 2016. Using a professional development program to enhance undergraduate career development and employability. International Journal of Innovation in Science and Mathematics Education (formerly CAL-laborate International). 24(3).

Considine J, Curtis K, Shaban R, Fry M. 2018. Consensus-based clinical research priorities for emergency nursing in Australia. Australasian Emergency Care. online ahead of print.

Dechartres A, Ravaud P. 2015. Better prioritization to increase research value and decrease waste. BMC medicine. 13(1):244.

Dennis A, Cleland J, Johnston P, Ker J, Lough M, Rees C. 2014. Exploring stakeholders’ views of medical education research priorities: a national survey. Medical Education. 48(11):1078-1091.

Frenk J, L C, Bhutta Z. 2010 Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet 376:1923-1958.

Gorman D. 2015. Developing health care workforces for uncertain futures. Academic Medicine. 90(4):400-403.

Heiman N, Davis R, Rothberg B. 2018. A deeper understanding of depression and suicidality among medical students. Medical Teacher. [online ahead of print]:DOI: 10.1080/0142159X.0142018.1467559.

Hodges B, Albert M, Arweiler D, Akseer S, Bandiera G, Byrne N, Charlin B, Karazivan P, Kuper A, Maniate J. 2011. The future of medical education: a Canadian environmental scan. Medical Education. 45(1):95-106.

Holligan C, Wilson M, Humes W. 2011. Research cultures in English and Scottish university education departments: an exploratory study of academic staff perceptions. . British Educational Research Journal. 37(4):713-734.

Jackson D. 2014. Factors influencing job attainment in recent Bachelor graduates: evidence from Australia. Higher Education,. 68(1):135-153.

Jackson D, Collings D. 2018. The influence of Work-Integrated Learning and paid work during studies on graduate employment and underemployment. Higher Education 76(3):403-425.

Kernan W, Wheat M, Lerner B. 2008. Linking learning and health: A pilot study of medical students’ perceptions of the academic impact of various health issues. Academic Psychiatry. 32(1):61-64.

Limniou M, Smith M. 2010. Teachers’ and students’ perspectives on teaching and learning through virtual learning environments. European Journal of Engineering Education. 35(6):645-653.

Monterosso L, Ross-Adjie G, Keeney S. 2015. Developing a research agenda for nursing and midwifery: a modified Delphi study. Contemporary Nurse. 51(1):83-95.

Rankin G, Rushton A, Olver P, Moore A. 2012. Chartered Society of Physiotherapy's identification of national research priorities for physiotherapy using a modified Delphi technique. Physiotherapy. 98(3):260-272.

Ricketts T, Fraher E. 2013. Reconfiguring health workforce policy so that education, training, and actual delivery of care are closely connected. Health Affairs. 32(11):1874-1880.

Ritchie J, Spencer L. 1994. Qualitative data analysis for applied policy research. In: Bryman A, Burgess B, editors. Analyzing qualitative data. Routledge; p. 187-208.

Slavin S, Schindler D, Chibnall J. 2014. Medical student mental health 3.0: improving student wellness through curricular changes. Academic Medicine. 89(4):573-577.

Tootoonchi M, Yamani N, Changiz T, Yousefy A. 2012. Research priorities in medical education: A national study. Journal of research in medical sciences: the official journal of Isfahan University of Medical Sciences. 17(1):83.

VanderLind R. 2017. Effects of Mental Health on Student Learning. Learning Assistance Review. 22(2).

Varker T, Metcalf O, Forbes D, Chisolm K, Harvey S, Van Hooff M, McFarlane A, Bryant R, Phelps A. 2018. Research into Australian emergency services personnel mental health and wellbeing: an evidence map. Australian and New Zealand Journal of Psychiatry. 52(2):129-148.

Viergever R, Olifson S, Ghaffar A, Terry R. 2010. A checklist for health research priority setting: nine common themes of good practice. Health Research Policy and Systems. 8(1):36.

Webster F, Krueger P, MacDonald H, Archibald D, Telner D, Bytautas J, Whitehead C. 2015. A scoping review of medical education research in family medicine [journal article]. BMC Medical Education. 15(1):79.

Wilkinson T, Weller J, McKimm J, O'connor B, Pinnock R, Poole P, Sheehan D, Tweed M, Wearn A. 2010. Programmatic research in medical education: a national collaboration. The New Zealand Medical Journal 123:1318.

Worley P, Schuwirth L. 2014. Opinion versus value; local versus global: what determines our future research agenda? Medical Education. 48(11):1040-1042.

Figure 1: Sequential mixed methods study design.

Table 1. Characteristics of participants in stages 1, 2 and 3.

Characteristics

Stage 1

n=104

Stage 2

n=780

Stage 3

n=16

Age (years)

15 – 20

21 – 30

31 – 40

41 – 50

51 – 60

61+

Missing

2 (2%)

13 (13%)

18 (17%)

23 (22%)

24 (23%)

8 (8%)

16 (15%)

Median=

27 years

(IQR=22-45)

0

2 (13%)

3 (19%)

5 (30%)

3 (19%)

2 (13%)

1 (6%)

Gender

Female

Male

Missing

65 (62%)

29 (28%)

10 (10%)

458 (59%)

154 (20%)

168 (21%)

12 (75%)

4 (25%)

0

Ethnicity

Anglo-Saxon*

Non-Anglo**

Missing

88 (84%)

7 (7%)

9 (9%)

393 (50%)

221 (28%)

166 (21%)

15 (94%)

1 (6%)0

Stakeholder

Student

Researcher

Patient/client

Educator/clinician

Missing

13 (13%)

13 (13%)

19 (18%)

51 (49%)

8 (7%)

299 (38%)

31 (4%)

59 (8%)

269 (34%)

122 (16%)

3 (19%)

4 (25%)

4 (25%)

5 (31%)

Group

Health Sciences***

Medicine

Nursing and/or Midwifery

Allied Health****

Other

Not applicable

Missing

10

16

9

42

1

8

18

116 (15%)

159 (20%)

44 (6%)

288 (37%)

36 (5%)

1 (0.1%)

136 (17%)

2 (12.5%)

2 (12.5%)

2(12.5%)

6 (37.5%)

0

4 (25%)

0

* Australia, New Zealand, UK, the Americas or Europe

**Africa, Middle East, Asia

*** biomedical science, nutrition science, pharmaceutical science, psychological science, health science

****dietetics, occupational therapy, paramedicine, pharmacy, physiotherapy, radiography, radiation therapy, social work, sonography

Table 2. Priorities as themes and subthemes identified from the stage 1 survey

Overarching Theme

Subtheme

Definition

Priorities focused on the trainee/student

Effectively select/recruit healthcare students to reflect needs

Relates to how students are selected and retained to adequately support workforce planning needs

Understanding the factors influencing career choice

Examines issues such as recruiting students/trainees for specific specialties and/or professions in Australia and the role that issues such as gender, generation, educational opportunities and culture might play

Understanding how to promote resiliency and well-being

Understanding how to promote resiliency and well-being in students, including supporting those from diverse backgrounds as they progress through their training

Priorities focused on the educators

Understanding how to better select/approve educators*

Addresses concerns about the importance of hiring educators who have expressed interest in teaching and also have experience

Understanding how to support/value the role of educators*

Encompasses the many aspects of support an individual needs in order to successfully work as an educator. These may include abstract support such as valuing the role of educator or more concrete aspects of support such as ensuring protected teaching time (or providing practical resources for teaching)

Understanding how to facilitate faculty development

Relates to the idea that it is essential to ensure that trainers are given opportunities to develop their skills, knowledge, behaviours and practices as educators

Working with others in the workplace environment

Understanding how to promote teamwork

Concerns about the challenges that must be negotiated by healthcare teams in order to effectively and efficiently work together

Understanding how we foster interprofessionalism

The need to address the difficulties and challenges to staff in which many professionals must work together as a team

Understanding how leadership should be developed

Concerns teaching and promoting leadership capabilities

The culture of workplace learning environments

Understanding how to balance education /service conflicts

Concerns the pressures that exist or are perceived to exist between the delivery of service to patients (and patient outcomes) and the provision of education

Creating an effective workplace learning culture

Concerns issues around the workplace learning environment and exploring how the effectiveness of that environment can be enhanced

Curriculum integration

Understanding the roles of the informal/formal curricula

This theme refers to examining the integration of workplace learning and theoretical aspects learnt in the classroom

Vertically integrating the undergraduate and post graduate curricula*

Concerns considering education as a lengthy continuum and ensuring that undergraduate study is linked all the way through the final years of training and beyond

Campus based learning^

This theme relates to the importance of campus based, including online learning and how it can be improved.

Workplace-based learning^

This theme relates to the importance of work- based learning and how it can be improved

Curricular content

Understanding how to enhance communication skills

An important aspect of being a successful healthcare professional is being able to communicate effectively not only with patients but also with colleagues

Understanding how professionalism develops

This concerns issues that surround teaching, learning, and assessing professionalism, specifically aiming to develop professionalism in healthcare students and trainees as they move through their training

Ensuring the curriculum is fit for purpose

This theme relates to the idea of ensuring the current curriculum adequately prepares healthcare students for practice and is fit for purpose

Understanding how best to ensure students develop the required clinical skills for practice

This relates to clinical and procedural skills and clinical reasoning and problem solving

Understanding how we teach evidence based practice

Includes evidence based practice and research skills

Population/global health^

This topic concerns students understanding population and global health needs and helping their ability to become global health practitioners

Indigenous health^

This topic concerns students understanding how to work in Indigenous cross-cultural context

Rural/remote health^

This topic concerns students understanding rural and remote health issues and needs

Curriculum delivery

Understanding how technology impacts healthcare education

This topic concerns the idea that new educational technologies (incl. blended learning) are constantly emerging and it is essential that healthcare educators are able to successfully incorporate these new technologies into education

Understanding the role of simulation in training

This relates to the idea of examining and exploring how simulation should be integrated into the current curriculum and looking at issues such as the impact of simulation training on long term improvements

Understanding how best to teach specific topics and the most effective teaching methods / mode of learning

This relates to how best to educate students in specific topics and the most effective teaching methods and triggering curriculum change as a result. Including: Tailoring teaching needs to individuals learning needs; understanding the effect of internationalisation on teaching methods; work-integrated learning methods; and the role of peer assisted learning

Cost effectiveness^

This theme relates to the cost effectiveness of educational delivery

Assessment/feedback

Better understanding of the role of consistent and robust assessments / assessment approach

There have been concerns raised about the value of assessments. Stakeholders are calling for more evidence surrounding these assessments and understanding of how they can be more successfully implemented

Understanding how to provide more useful feedback

This relates to understanding how to successfully give feedback to trainees that is useful, but also examining how trainees respond to and utilise feedback

Ensuring assessments lead to fitness for practice

Concerns whether current assessments successfully capture the skills and abilities that students will need in order to be fit to practice. Includes the need to standardise marking and assess underperformance

* Priority identified in previous studies (Dennis et al. 2014; Ajjawi et al. 2017) but not in stage 1 of this study

^ Priorities identified only in this study, not previously identified

Table 3: 30 Priorities identified by rank and Likert-rating scale

Priority

Median n=780

IQR

Rank order sum n=629


5

4-5

472

Understanding how to promote resiliency and well-being in students

5

4-5

423

Ensuring the curriculum prepares is fit for purpose

5

4-5

419


5

4-5

219

Understanding the most effective teaching methods/modes of learning for different topics

4

4-5

201


5

4-5

187

How to effectively select/recruit healthcare students to reflect healthcare needs in Victoria

4

4-5

184

Understanding how to better select/approve educators

4

4-5

179

Understanding how to support/value the role of educators

4

4-5

146


4

4-5

131


4

4-5

130

Understanding how to provide more useful feedback

5

4-5

118

Understanding the role of workplace-based learning (i.e. clinical placements) as part of health-related curricula

5

4-5

101


4

3-5

96


4

4-5

87

Understanding how to balance education/service conflicts

4

3-5

72

Vertically integrating the undergraduate curricula with postgraduate learning

4

3-5

68

Understanding the role of consistent and robust assessments/assessment approaches

4

4-5

64

Understanding how we teach evidence-based practice

4

4-5

62


4

3-5

57


4

3-5

53


4

3-5

51

Teaching and learning of Indigenous health

4

3-5

46

Teaching and learning of population health/global health

4

3-5

43


4

3-5

39

Teaching and learning of rural/remote health

4

3-5

27


4

3-5

25

Understanding how to facilitate faculty development

4

3-5

20

Understanding the role of campus-based learning as part of health-related curricula

4

3-5

16

Understanding the cost-effectiveness of curriculum delivery

4

3-5

10

IQR=interquartile range

Table 4: Pattern matrix from Principle Component Analysis with direct oblimin rotation (n=780)

Description of priority

F1

F2

F3

F4

F5

F6

h2

rit

Understanding how we foster inter-professionalism

0.841

0.710

0.644


0.832

0.712

0.652


0.806

0.659

0.615


0.593

0.556

0.607


0.590

0.575

0.646

Understanding how to balance education/service conflicts

0.412

0.483

0.620


0.401

0.525

0.630

Understanding how to facilitate faculty development*

0.364

0.575

0.617


0.685

0.578

0.498


0.704

0.653

0.535


0.669

0.621

0.507

Understanding the role of workplace-based learning as part of health-related curricula*

0.486

0.613

0.557

Understanding how to provide more useful feedback*

0.464

0.594

0.594

Understanding how we teach evidence-based practice

0.355

0.528

0.629


0.764

0.685

0.617

Teaching and learning of Indigenous health

0.928

0.826

0.570

Teaching and learning of rural/remote health

0.920

0.839

0.616


0.538

0.575

0610


0.319

0.468

0.589


0.516

0.565

0.595

Understanding the cost-effectiveness of curriculum delivery

0.512

0.563

0.555


0.682

0.691

0.591

Understanding how to provide more useful feedback*

0.397

0.594

0.594


0.549

0.435

0.480


0.599

0.592

0.574


0.593

0.578

0.530


0.579

0.523

0.505

Understanding how to better select/approve educators

0.762

0.657

0.560

Understanding how to facilitate faculty development*

0.375

0.575

0.617


0.555

0.596

0.625

Vertically integrating the undergraduate curricula with postgraduate learning

0.715

0.626

0.554

Understanding the role of campus-based learning as part of health-related curricula

0.661

0.656

0.633

Understanding the role of workplace-based learning as part of health-related curricula*

0.466

0.613

0.557

Eigenvales

11.59

1.834

1.375

1.312

1.134

1.005

Explained variance

38.65

6.11

4.58

4.37

4.78

3.35

Cronbach

0.88

0.82

0.86

0.78

0.79

0.77

Notes: *items asterisked and italicised loaded on two factors; h2=commonalities; rit=corrected item-total correlation.

F1= Culture of learning together in the workplace; F2=Preparation for work; F3=Meeting future Australian healthcare needs^; F4=Pedagogical Effectiveness; F5=Workforce issues; F6=Curriculum integration.

^Priorities related to population groups with high health care needs in Australia.

Table 5. Reasons provided by respondents about why these should be priorities for health education research in decreasing order of frequency

Theme

Description of theme

Priorities with which the reason is most commonly associated

Priority and illustrative quote

Improved student learning

This is concerned with healthcare professions education research being carried out with the aim of improving student learning (e.g. measuring the impact of clinical placements on student learning)


Understanding how to best ensure students develop the required clinical skills for practice

Understanding how to better select and approve educators


“We need to set a context for our own students and while there are a lot of similarities with students around the world, we also need to recognise there are differences that impact on their learning. A greater understanding of these can help us to develop courses that best suit the needs of our students” (Female Biomedical Science Educator [Stage 1 #61])

Graduate employability or preparing for career*

This is concerned with malalignment between the supply and demand of healthcare and health sciences professionals and also with the extent to which students are equipped for the workforce (e.g. job prospects for a recently graduated biomedical sciences student)

Ensuring the curriculum prepares for practice/is fit for purpose



Ensuring the curriculum prepares for practice or is fit for purpose

“Because without appropriate curriculum for practice, I worry about not being prepared and equipped to gain employment or succeed in my employment” (Female Radiation Therapy Student [Stage 2 #56])

Current problem

This is concerned with a perceived problem area of current practice, education or need (e.g. current issues in the healthcare system which could be addressed with educational interventions)





“There is much discrepancy across and within healthcare facilities amongst the clinical educators as to the scope of their role, and their ability to manage student deficits …” (Female Nursing Academic [Stage 1 #79])

Faculty safety and well-being

This is concerned with why certain topics should be prioritised because of their importance to faculty safety and well-being (e.g. ensuring sufficient numbers of clinical staff to manage patient care and student education requirements)





“Because I read a moving piece from the wife of a doctor who committed suicide” (Female Patient [Stage 1 #90])

Student safety and well-being

This is concerned with why certain topics should be prioritised because of their importance to student safety and well-being (e.g. stress and burnout among students)





“The current generation of health professional students going through university courses are prone to high frequencies of stress, anxiety and related issues” (Male Occupational Therapy Educator [Stage 1 #42])

Patient safety

One of the themes as to why certain topics should be prioritised relates to their importance in improving patient safety (e.g. ensuring healthcare students attain the required competences leads to increased patient safety)





“During placement it was difficult to interpret or seek out information from some of the patient notes. This was due to vague notes, the use of terminology that maybe foreign to another healthcare professionals … This was also hard to follow when looking for patient assessments and treatment regimes.” (Male Nursing Student [Stage 1 #72])

Other

Other reasons for selecting a priority (e.g. it is an important area for research or applies to the broader population or context)





“Communication will help bring new social connections, allowing a greater range of interactions and form new relations.” (Female Nutrition Science Student [Stage 2 #363])

Evidence-based education

This is concerned with the idea that issues should be examined further if there is little experimental evidence to say that a current approach is the best solution or way to proceed and/or is cost-effective (e.g. lack of evidence supporting a pedagogical approach)





“Assessments are many and varied over the courses. Understanding which assessments and approached work best for student development is critical.”

(Female Educator [Stage 2 #18])

Investing in the Future

This is concerned with the idea that certain priorities need to be addressed in order for healthcare education to be successful in the future and the concept that by focusing on students, we are investing in the future (e.g. ensuring the healthcare workforce is equipped to meet increasing population health needs)





“We all need general practitioners - will there be enough locally trained ones or will we be overrun by specialists in the cities?” (Female Patient [Stage 1 #41])

Quality of Care

One of the themes as to why certain topics should be prioritised relates to quality of care that is care over and above safe care (e.g. ensuring that approaches to teaching and learning leads to the provision of high quality care)


Understanding how to foster interprofessionalism



“A number of students have attended in the last few years who have not engaged in appropriate professional communication with patients …Perhaps there is an opportunity to see how this could be done better, to investigate the perceptions of students around professional and empathetic communication, or in the way this is taught, or their preparedness for communicating in the clinical setting.” (Female Radiation Therapy Educator [Stage 1, #51])

Personal interest/challenge

This is concerned with healthcare education research being carried out for personal interest or personal challenge, including impact (e.g. interest in mental health issues due to past personal experience)



Understanding the factors affecting career choice


“Because my role is to facilitate an understanding of basic anatomy and physiology for first year allied health students - to form a knowledge framework/foundation for their vocational learning. I want to know what is the most effective methods of teaching and learning the various topics covered in my units.” (Female Nutrition Science Educator [Stage 2 #84])

This priority addresses one or more of the other priorities*

This was considered when participants stated that addressing one priority would also address one or more others (e.g. if students have good mental health they will have greater capacity to learn)




Ensuring the curriculum prepares for practice or is fit for purpose

“In health disciplines practice is the objective. Safe and effective practice is the main end goal. Many of the other topics fold in underneath this but this is the outcome that is most worth researching and measuring.” (Female Medicine Educator [Stage 2 #524])

Policy/political agenda

Another reason items have been prioritised is that they are a part of the larger policy or political agenda (e.g. new legislation or a change in Government)


How to effectively select/recruit healthcare students to reflect healthcare needs



“I definitely think it is important for the future of our population’s health, to target areas of need e.g. rural/regional practice, populations of need, migrants, refugees and asylum seekers … although this is more of a state/national issue, rather than a university level issue” (Female Medicine Student [Stage 2 #242]

Note: quotes were often coded to multiple themes (i.e. reason for priority)

*New reason/theme identified in Stage 2

2

eprints.whiterose.ac.ukeprints.whiterose.ac.uk/147062/1/manuscript_revised... · web viewthe total...

Documents