|
| |
 |
|
| ORIGINAL ARTICLE |
|
| Year : 2021 | Volume
: 26
| Issue : 3 | Page : 103-109 |
|
Assessing a research training programme for rural physicians
Cameron MacLellan1, Cheri Bethune2, Thomas Heeley3, Wendy Graham2, Cathryn Button4, Shabnam Asghari5
1 Centre for Rural Health Studies, M5M105, Medical Education Building, Health Sciences Centre, Newfoundland, Canada
2 Department of Family Medicine, Faculty of Medicine, Memorial University of Newfoundland, Health Sciences Centre, Newfoundland, Canada
3 Centre for Rural Health Studies, M5M105, Medical Education Building, Health Sciences Centre; Department of Family Medicine, Faculty of Medicine, Memorial University of Newfoundland, Health Sciences Centre, Newfoundland, Canada
4 Department of Psychology, Memorial University of Newfoundland, Newfoundland, Canada
5 Centre for Rural Health Studies, M5M105, Medical Education Building, Health Sciences Centre; Department of Family Medicine, Faculty of Medicine, Memorial University of Newfoundland, Health Sciences Centre; Primary Healthcare Research Unit, Agnes Cowan Hostel, Health Sciences Centre, St. John's, Newfoundland, Canada
| Date of Submission | 07-Jun-2020 |
| Date of Decision | 31-Jul-2020 |
| Date of Acceptance | 14-Oct-2021 |
| Date of Web Publication | 02-Jul-2021 |
Correspondence Address:
MD, MPH, PhD Shabnam Asghari
Centre for Rural Health Studies, M5M105, Medical Education Building, Health Sciences Centre; Department of Family Medicine, Faculty of Medicine, Memorial University of Newfoundland, Health Sciences Centre; Primary Healthcare Research Unit, Agnes Cowan Hostel, Health Sciences Centre, St. John's, Newfoundland
Canada
 Source of Support: None, Conflict of Interest: None
DOI: 10.4103/CJRM.CJRM_53_20
Introduction: To assess the effect of a training programme called 6for6 (the programme) on research competency and productivity amongst rural physicians. The programme develops the research skills of six rural physicians over six weekends. Physicians learn about various research methods and writing techniques through blended learning components.
Methods: We conducted a quasi-experimental study, comparing research competency and productivity between intervention and non-equivalent control groups and over time through a repeated measures design. Generalized linear mixed model (GLMM), ANOVA, and Cochran Q tests were conducted. The intervention was provided to five groups of 6 rural physicians each between 2014 and 2019. Main outcome measures: self-assessed research competency (knowledge, attitudes and skills) and productivity (publications, grants and presentations of research-related work at conferences) were our primary and secondary outcomes, respectively. We measured the outcomes before, during and after the programme. Controls: Rural physicians who expressed interest in the programme and later enrolled.
Results: This study shows that, amongst its thirty participants, overall research competency was significantly different between intervention and control groups (65.7% ± 37.6% and 58.6% ± 14.4%, P < 0.05 for GLMM). The percentage of participants who were productive before, during and after the programme was 26.7%, 16.7% and 50.0%, respectively. Overall, productivity rates were significantly different between intervention and control groups (rate difference was 72.2/100 person-years, P < 0.05 for GLMM).
Conclusion: This study suggests that the programme improves research competency and productivity for rural physicians. Rural physicians who wish to improve their research competency would benefit from participating in similar programmes.
Résumé
Introduction: Évaluer l'effet d'un programme de formation intitulé 6for6 (le programme) sur les compétences en recherche et la productivité parmi les médecins des régions rurales. Le programme permet à six médecins en région rurale d'acquérir des compétences en recherche durant six fins de semaine. Les médecins apprennent diverses méthodologies de recherche et techniques de rédaction par l'entremise d'un programme d'apprentissage mixte.
Méthodologie: Nous avons réalisé une étude quasi-expérimentale, qui comparait les compétences en recherche et la productivité entre des groupes non-équivalents intervention et témoin, et dans le temps, par une méthodologie à mesures répétitives. Un modèle linéaire à effets mixtes généralisé (GLMM), un modèle d'analyse de variance, et des tests Q de Cochran ont été réalisés. L'intervention a été appliquée à 5 groupes de 6 médecins en région rurale, entre 2014 et 2019 dans tous les cas. Paramètre d'évaluation: compétences en recherche évaluées par l'apprenant (connaissances, attitudes et compétences) et productivité (publications, subventions et présentation des travaux de recherche aux congrès) étaient respectivement nos paramètres d'évaluation principal et secondaire. Nous avons mesuré les paramètres avant, durant et après le programme. Les médecins en région rurale ayant manifesté de l'intérêt à l'égard du programme ont été inscrits.
Résultats: L'étude montre que parmi les 30 participants, les compétences générales en recherche étaient significativement différentes entre les groupes intervention et témoin (65,7 ± 37,6% et 58,6 ± 14,4%, P < 0,05 pour le GLMM). Le pourcentage de participants qui étaient productifs avant, durant et après le programme était respectivement de 26,7, 16,7 et 50,0%. Dans l'ensemble, la productivité était significativement différente entre les groupes intervention et témoin (différence des taux: 72,2 par 100 années-personnes, P < 0,05 pour le GLMM).
Conclusion: Cette étude laisse penser que le programme améliore les compétences en recherche et la productivité chez les médecins en région rurale. Les médecins en région rurale qui souhaitent améliorer leurs compétences en recherche bénéficieront de programmes semblables.
Mots-clés: Compétences en recherche, programme de formation en recherche, santé en région rurale, communautés rurales
Keywords: Research skills, research training programme, rural communities, rural health
How to cite this article:
MacLellan C, Bethune C, Heeley T, Graham W, Button C, Asghari S. Assessing a research training programme for rural physicians. Can J Rural Med 2021;26:103-9 |
How to cite this URL:
MacLellan C, Bethune C, Heeley T, Graham W, Button C, Asghari S. Assessing a research training programme for rural physicians. Can J Rural Med [serial online] 2021 [cited 2023 Jun 6];26:103-9. Available from: https://www.cjrm.ca/text.asp?2021/26/3/103/320530 |
| Introduction | |  |
For most rural physicians, engaging in scholarship is challenging. As described in the CanMEDs framework, those who wish to participate in research must pursue 'advanced research training'.[1] While research training is provided during undergraduate medical education and residency training, studies suggest that it receives limited curricular time.[2],[3] Furthermore, advanced research training programmes are not accessible for rural physicians once in practice due to geographical and professional isolation and a lack of time and funding.[4],[5],[6],[7],[8],[9]
Rural physicians are often interested in exploring questions related to their clinical practice[8],[9] and bring an important contextual understanding of rural communities to bear on health-care research.[2],[10],[11],[12],[13 Given the geographical diversity between rural communities and a gap in rural health-care research, rural physicians have potential to develop research that yields locally feasible solutions.[10]
Although research training programmes do improve research activities amongst health-care professionals,[14],[15] our literature search found that a limited number of programmes are available to support rural physicians' research endeavours in a variety of settings. The clinician–scholar support team in Japan provides online research support for rural physicians,[16] while a few programmes in Australia provide research support either in urban or rural settings.[17],[18],[19],[20] Furthermore, these programmes provide limited support for rural physicians' research activities,[16],[17] and only certain authors have published assessments of programme outcomes such as research competency and productivity.[18],[19],[20] In research, competency is a subjective measure of the relationship between knowledge, attitudes and skills of an individual that combine to produce results.[21] Research productivity often takes the form of publications, grants or presentations of research-related work at conferences and is regarded as an objective measure of research competency.[22]
To empower rural physicians to pursue their research interests, Memorial University of Newfoundland developed a research training programme called 6for6 (the programme).[23],[24] This 1-year programme focused on developing the research capabilities of 6 rural physicians, taking place through face-to-face sessions over 6 weekends (Friday and Saturday only). The purpose of this study is to assess the effectiveness of the programme in building research competency (knowledge, attitudes and skills) and productivity (publications, grants and presentations of research-related work at conferences) amongst its participants.
| Methods | | |
Study design
This quasi-experimental study occurred from April 2014 to October 2019 at Memorial University of Newfoundland, comparing research competency and productivity between intervention and non-equivalent control groups and through a repeated measures design.
Intervention
The intervention in this study is the programme, which assists rural physicians to develop research capabilities. Through a blended learning curriculum, participants learn research methods and writing techniques, develop their own research projects with a mentor and cultivate a research network with other rural physicians. They are also supported by the programme coordinator.[24] We delivered the programme to 5 different groups of 6 rural physicians each year from 2014 to 2019 inclusive.
Study population and inclusion criteria
Rural physicians practising in Newfoundland and Labrador, Nunavut and New Brunswick were eligible to apply. Candidates applied by submitting a letter of interest detailing a research idea related to their local practice, along with a resume and answers to eligibility screening questions. Participants were required to have at least 1 year of experience practising in a rural area. No previous research training was necessary. Participants with full-time faculty affiliations at any university were excluded from this study.
Outcome measures
The primary outcome is research competency, defined as participants' knowledge, attitudes and skills. Knowledge refers to participants' understanding of research concepts and their ability to recall the information. Attitudes represent the extent to which one views research as valuable and worthwhile. Participants' research skills refer to their ability to put research knowledge into practice.[21]
The secondary outcome, research productivity, refers to participants' publications, grants and presentations of research-related work at conferences. Any articles successfully published in a peer-reviewed journal or successful applications for research funding count as publications and grants, respectively. Presentations of research-related work at conferences refer to workshops or presentations (poster, oral or keynote) at local, national or international research symposia.[22]
Non-equivalent control groups
The control groups were recruited from the pool of rural physicians who expressed interest in the programme and later enrolled. By the time of first contact with participants, they had not received any prior research training. For every individual who received the intervention, we used up to four controls. All participants reported baseline values before programme entry, which allowed us to compare the intervention group of 1 year with control groups represented by those enrolled in different years. For example, individuals who received the intervention in year 5 were compared to the controls in years 1–4, while those in year 4 were compared to the controls in years 1–3 and 5.
Data collection
Each year we measured participants' self-assessed research competency and productivity at zero months, during the programme and at 12 months using the same survey. The pre-programme survey was collected at zero months, the interim survey was collected during the programme, and the post-programme survey at 12 months. To measure research competency during the programme, we divided the competency survey into six sections and delivered them 1 week after each session; each section corresponded with the curricular topics of each session. We combined these survey sections to create the interim-programme survey.
Data collected before the programme represented physicians' research competency at baseline and thus established the control group, while data collected at 12 months represented the intervention group.
The generalised linear mixed model (GLMM) allowed us to compare the intervention group of 1 year to the control groups of all other years until each year had a chance to represent the intervention group. This approach allowed us to control for the effects of time. Since each group of participants enrolled in the programme in different years, we did not collect data for the intervention and control groups simultaneously.
Using the research productivity questionnaire, we collected data about participants' productivity before, during and after the programme. We conducted a respondent validation questionnaire in September 2019 to verify the accuracy and recency of this information. We used productivity data collected at the beginning and end of the programme (e.g., at zero and 12 months) to compare the control and intervention groups through a GLMM.
To improve response rates, we reminded participants three times to complete the surveys at 2-week intervals.
Data analysis
We performed descriptive analyses to assess response rates to the surveys and questionnaires and demographic characteristics of the participants.
To assess change in research competency over time, we used a two-way, repeated measures ANOVA where we compared the mean differences between scores in the pre-, interim- and post-programme surveys. We used GLMM to compare the post-programme survey scores of intervention groups with the pre-programme scores of control groups.
For research productivity, we conducted a repeated measures analysis using the Cochran Q test to determine changes over time (before, during and after the programme). To assess for differences in research productivity rates between intervention and control groups, we calculated the number of research products per 100 person-years and analysed the data using a GLMM.
We performed all analyses in R studio, with a P < 0.05 being considered significant. For both research competency and productivity, we controlled for differences within and between groups using the GLMM. We accounted for differences related to time by including years of practice in the R commands. This study was approved by the Newfoundland and Labrador Health Research Ethics Authority.
| Results | | |
During the 5-year study period, 30 rural physicians enrolled in the programme, and 19 (63.3%) were female. There were 27 (90.0%) physicians who practised in Newfoundland and Labrador and 3 (10.0%) from Nunavut. Approximately 83.3% (n = 25) were family physicians, while the remaining participants were from other specialities (n = 5, 16.7%). Research competency survey response rates were 100% for the pre-programme survey, 93.3% for the interim-programme survey, and 76.7% for the post-programme survey. When we ran the GLMM, the response rate for the control group was 100% and 76% for the intervention group. The response rate for the respondent validation questionnaire was 19 (63.3%). We included all participants in the analysis and assumed that non-respondents had no additional research activities since completing the research productivity questionnaire. No participants dropped out of the programme.
Effect of the programme on self-assessed research competency
The mean and standard deviation for the pre-, interim-, and post-programme questionnaire scores for overall competency were 58.6% ± 14.6%, 61.1% ± 24.4% and 65.7% ± 37.6% respectively; we observed no significant differences between these scores through the repeated measures analysis. A summary of these results can be found in [Table 1], which also includes the results for research knowledge, attitudes and skills. | Table 1: Research competency scores of 6 for 6 participants who completed the pre , interim , and post programme surveys (n=30)
Click here to view |
The results of the GLMM showed differences in mean competency scores between the intervention and control groups [Table 2], which revealed a significant increase between the pre- and post-programme scores in overall research competency (mean and standard deviation: 58.6% ± 14.4% and 65.7% ± 37.6%, P < 0.05). | Table 2: Research competency scores of 6 for 6 participants in intervention and control groups (n=30)
Click here to view |
Effect of the programme on research productivity
[Table 3] shows the repeated measures results for all components of productivity. The results of the Cochran Q test demonstrate that the proportion of participants who published articles after the programme was significantly higher than before and during the programme (P < 0.05). | Table 3: Research productivity of 6 for 6 participants before, during, and after the 6 for 6 programme (n=30)
Click here to view |
Overall, the GLMM revealed a significant improvement in productivity rates between the control and intervention groups [Table 4]. The intervention group had significantly higher publication rates, rates of secured grants and presentations of research-related work at conferences. | Table 4: Research productivity rates of 6for6 participants in intervention and control groups (n=30)
Click here to view |
The sensitivity analysis for the productivity and respondent validation questionnaires showed that all results were consistent with the original data set.
| Discussion | | |
This study shows that the 6for6 programme increases rural physicians' research competency and productivity compared to the control groups. Our results are consistent with other studies.[16],[18] Although knowledge, skills, presentations of research-related work at conferences and grants increased by the end of the programme, the repeated measures analysis demonstrated that these results were not significant. This could be due to the small sample size of the study. For an example, the rural research capacity building programme in Australia found significant increases in research experience scores and publication rates with high sample sizes.[18],[19]
The sensitivity analysis found that results for competency and productivity were consistent in all categories except for attitudes. This is consistent with previous studies, which suggest that building positive attitudes toward research takes time.[25],[26] Study participants could possibly benefit from spending 2 years in the programme instead of one.
The availability of external research support could be a factor in research productivity outcomes. In this study, alumni who worked in the Labrador-Grenfell Regional Health Authority were eligible to apply for grant funding through an extension programme.[27] We conducted a sensitivity analysis by excluding those who were eligible for these grants (n = 4). Although the effect size of the productivity rate decreased, the results remained significant. This suggests that similar interventions are effective; however, additional support from an external source seems to contribute to an increase in research productivity.
Limitations
This quasi-experimental study using non-equivalent control groups should be interpreted in light of its limitations.
Some aspects of the programme's delivery limit our findings. While alumni who participated earlier during the study have had more time to produce research, those from later years may have benefitted from programme improvements. These improvements applied to the content delivered, session activities, daily schedule and personnel involved in the study. To control for these factors, we used a GLMM with random effects to compare research competency and productivity between groups. We found no significant differences. To further address this limitation, the programme established a 'Come Home Year', where previous participants were invited for a weekend retreat to reconnect with mentors and peers to discuss new and existing research projects.
The number of survey items increased over time, potentially influencing survey performance of participants from the final 3 years of the study. We controlled for the effect of time and found no significant differences between groups with different survey lengths.
One participant produced a high amount of research in comparison to the rest of the groups. While the literature suggests that this phenomenon is common for research training programmes,[18] there is potential that prolific research production from a single participant can skew the results. We conducted a sensitivity analysis by removing this participant and found no changes in the results.
Non-response bias is a limitation of this study due to incomplete surveys and questionnaires. To mediate this bias, we imputed data to test the consistency of the results with several scenarios (e.g., best- and worst-case scenarios). The details of the sensitivity analysis are available upon request.
Due to a small sample size and to ensure confidentiality, we could not control for variables such as gender/sex, speciality or years of practice. As a result, we were unable to match the intervention groups to the controls based on years of practice. Future research would also benefit from a larger sample size so that possible moderating influences such as sex, speciality and years of practice could be assessed.
The Hawthorne effect is another limitation, where participants were aware of their involvement in this research study and could potentially change their behaviour to affirm the hypothesis. There were several measurements during and after the study period; however, we did not see a shift in the findings over time.
Some tests may be significant due to multiple testing. We adjusted the P values in the repeated measures analysis and GLMM for research competency and productivity. All results remained significant except for overall research competency scores in the GLMM and the rate of presentations of research-related work at conferences per year.
Nonetheless, to fully assess the effect of experience with the passage of time, it is important to follow participants for a longer period of time. Statistical controls, while very useful, do not capture the myriad context effects that might occur in the multifaceted environment studied here.
Future research
Future research would benefit from a longer time frame to ensure participants have enough time to finish their research projects. This alternative option would allow participants to publish their work by the end of the study and enable researchers to use additional measures of productivity such as citation counts, first author publications or amount of grant money awarded. Future studies could compare the effectiveness of their research programmes to a virtual stream for rural physicians who prefer to learn from home. This could benefit participants who wish to reduce the amount of travel required to pursue research training.
| Conclusions | | |
Rural physicians lack the resources to develop as researchers. This study found that the 6for6 programme enhances research competency and productivity amongst rural physicians. Although overall research competency and productivity increased between the intervention and control groups, attitudes toward research remain inconclusive. This is the first programme in Canada that helps rural physicians conduct research in the communities they serve. Similar programmes could help rural physicians develop research projects relevant to their patients and practice.
Financial support and sponsorship: This study was financially supported by Faculty of Medicine at Memorial University.
Conflicts of interest: There are no conflicts of interest.
| References | | |
| 1. | Frank J, Snell L, Sherbino J, editors. CanMEDS 2015 physician competency framework. Ottawa: Royal College of Physicians and Surgeons of Canada; 2015.  |
| 2. | Turgeon J, Côté L. Qualitative Research in Family Medicine. An Inevitable Development. Can Fam Physician 2000 46:2171-2. |
| 3. | Bowman RC, Crabtree BF, Petzel JB, Hadley TS. Meeting the challenges of workload and building a practice: The perspectives of 10 rural physicians. J Rural Health 1997;13:71-7. |
| 4. | Miedema B, Hamilton R, Fortin P, Easley J, Tatemichi S. The challenges and rewards of rural family practice in New Brunswick, Canada: Lessons for retention. Rural Remote Health 2009;9:1141. |
| 5. | Birden HH. The researcher development programme: How to extend the involvement of Australian general practitioners in research? Rural Remote Health 2007;7:776. |
| 6. | Curran VR, Fleet L, Kirby F. Factors influencing rural health care professionals' access to continuing professional education. Aust J Rural Health 2006;14:51-5. |
| 7. | Jarvis-Selinger S, Liman Y, Stacy E, Bluman R, Ho K, Abizadeh J. An assessment of British Columbia rural physicians' continuing professional development needs. B C Med J 2009;51:250-5. |
| 8. | Hutten-Czapski P. The rural scholar: Anathema or archetype? Can J Rural Med 2016;21:3-4. |
| 9. | Bethune C, Asghari S, Godwin M, McCarthy P. Finding their voices: How a group of academic family physicians became writers. Can Fam Physician 2014;60:1067. |
| 10. | Pong RW, Atkinson AM, Irvine A, MacLeod M, Minore B, Pegoraro A, et al. Rural health research in the Canadian Institutes of Health Research. A position paper prepared for Canadian Health Services Research Foundation and Social Sciences and Humanities Research Council. Sudbury, ON: Laurentian University, Lakehead University, University of Northern BC; 2000. |
| 11. | Murphy E, Mattson B. Qualitative research and family practice: A marriage made in heaven? Fam Pract 1992;9:85-91. |
| 12. | Reid AJ. What we want: Qualitative research. Promising frontier for family medicine. Can Fam Physician 1996;42:387-9, 397-400. |
| 13. | McLean R, Mendis K, Harris B, Canalese J. Retrospective bibliometric review of rural health research: Australia's contribution and other trends. Rural Remote Health 2007;7:767. |
| 14. | Workman M, Albert AY, Norman WV. Training family physicians as researchers: Outcomes over 15 years for Canada's first clinician scholar programme. Can Fam Physician 2019;65:45-51. |
| 15. | Rosser W, Godwin M, Seguin R. Family medicine research capacity building: Five-weekend programmes in Ontario. Can Fam Physician 2010;56:e94-100. |
| 16. | Matsubara S, Ohkuchi A, Kamesaki T, Ishikawa S, Nakamura Y, Matsumoto M. Supporting rural remote physicians to conduct a study and write a paper: Experience of Clinical Research Support Team (CRST)-Jichi. Rural Remote Health 2014;14:2883. |
| 17. | Schmidt DD, Kirby S. modular approach to rural and remote research education: A project report. Rural Remote Health 2016;16:3609. |
| 18. | Duncanson K, Webster E, Schmidt DD. Impact of a remotely delivered, writing for publication programme on publication outcomes of novice researchers. Rural Remote Health 2018;18:4668. |
| 19. | Schmidt DD, Webster E, Duncanson K. Building research experience: Impact of a novice researcher development programme for rural health workers. Aust J Rural Health 2019;27:392-7. |
| 20. | McIntyre E, Brun L, Cameron H. Researcher development programme of the primary health care research, evaluation and development strategy. Aust J Prim Health 2011;17:114-21. |
| 21. | Dilmore TC, Moore DW, Bjork Z. Developing a competency-based educational structure within clinical and translational science. Clin Transl Sci 2013;6:98-102. |
| 22. | Durieux V, Gevenois PA. Bibliometric indicators: Quality measurements of scientific publication. Radiology 2010;255:342-51. |
| 23. | McCarthy P, Bethune C, Fitzgerald S, Graham W, Asghari S, Heeley T, et al. Needs assessment for development of 6for6: Longitudinal research skills programme tailored to rural and remote family physicians. Can Fam Physician 2016;62:e80-8. |
| 24. | McCarthy P, Bethune C, Fitzgerald S, Graham W, Asghari S, Heeley T, et al. Curriculum development of 6for6: Longitudinal research skills programme for rural and remote family physicians. Can Fam Physician 2016;62:e89-95. |
| 25. | Siemens DR, Punnen S, Wong J, Kanji N. survey on the attitudes towards research in medical school. BMC Med Educ 2010;10:4. |
| 26. | Goldenberg JZ, Burlingham BS, Guiltinan J, Oberg EB. Shifting attitudes towards research and evidence-based medicine within the naturopathic medical community: The power of people, money and acceptance. Adv Integr Med 2017;4:49-55. |
| 27. | Asghari S, Heeley T, Walsh A, Rourke J, Bethune C, Graham W. Rural360: Incubating socially accountable research in the Canadian North. Int J Circumpolar Health 2019;78:1633191. |
[Table 1], [Table 2], [Table 3], [Table 4]
|
Search Pubmed for