Using E-mail based Continuing Medical Education for Family Physicians — Can it work?*
J. Neil Marshall MB CCFP*, Pamela J. Brett MA†, Moira A Stewart BSc,PhD‡, Truls Østbye MD,MPH§, * Thames Valley Family Practice Research Unit and University of Western Ontario Department of Family Medicine, †University of British Columbia Department of Family Medicine, ‡ Centre for Studies in Family Medicine, Family Medicine and Epidemiology and Biostatistics, University of Western Ontario, §Epidemiology and Biostatistics at the University of Western Ontario
Abstract: Objective: In 1994, the Family medicine Education and Research Network (FERN) was developed to support on-line discussion among London and area family physicians. FERN-D (FERN Dissemination) was introduced as a closed sub-group of FERN to support educational case discussions focused on prevention. The present study reports results of a pilot FERN-D intervention. Design: Pre- and Post- intervention mail-out surveys combined with an e-mail feedback questionnaire and a modified focus group. Setting: Rural and urban family physicians in the London Ontario area. Participants: Initial survey mailed to all area family physicians (N=480). 40 volunteer physicians recruited to the intervention group. Response rate was 50.4% to the pre-survey, and 68% to the follow-up survey (only mailed to responders) Main findings: A higher percentage of the FERN-D group indicated accurate knowledge for seven of the items than the comparison group. Both groups improved their preventive practice scores from pre- to post intervention. The majority of the FERN-D group felt that they had become aware of new techniques or relevant research (58.8%) and had made changes in their practice (64.7%). In both the e-mail feedback and the modified focus group convenience (89.2%) and interaction with family physician colleagues (86.5%) were cited as advantages to this method of CME. Conclusion: On-line case-based discussion is a promising strategy for enhanced research transfer to family physicians: more research is needed to determine if it can be effectively utilized to change physician behavior.
How can family practice be improved so that research results and guidelines are incorporated? Most traditional Continuing Medical Education (CME) approaches such as didactic lectures and conferences have not worked very well.1 More promising strategies for changing physician behavior include reminder systems, academic detailing, using peers as teachers, and patient-mediated strategies.2-4
Problem-based small group CME also appears promising.5 Key elements to practice change may include: surprise, conflict or novelty in the educational approach, the need for reinforcement over time, and perhaps trial of the change with a few patients.6,7
We believe that Internet-based e-mail discussion groups may provide these key elements. Physicians have the opportunity to discuss the material presented among themselves, reinforcing ideas over a much longer period than is usual for CME and possibly allowing trials of practice changes before the discussion of a topic ends.8,9
The purpose of this study was to pilot, test and evaluate an Internet-based e-mail discussion of research based cases by a group of family physicians and to describe any resultant changes in knowledge and practice.
Method
Setting - In 1994, the Thames Valley Family Practice Research Unit (TVFPRU; London, Ontario) developed of a local, Internet-based e-mail discussion group called the Family medicine Education and Research Network (FERN). The TVFPRU conducts research relevant to primary care in partnership with family physicians and other health care providers. FERN was established to provide a forum for London and area family physicians to discuss issues relevant to family practice, to strengthen the partnership through an increased sense of community and to raise the awareness of family medicine research. By September 1996, about a quarter of all area family physicians were members of FERN (129/480). Discussions on FERN are unstructured and largely unmoderated; topics ranged from discussion of political topics to patient care and family practice management issues.
Design - A pre- and post- design was used, including comparison of FERN-D Group participants with Comparison Group physicians (which included both the remaining FERN members and the remaining area family physicians.)
Participants - In the fall of 1996, 40 family physicians were recruited from the FERN members, to participate in a closed, moderated, e-mail CME group. Our recruitment method used e-mail messages and requests in a local research newsletter. There was no attempt to randomize the participants in this pilot study as we wanted the participants to have knowledge of e-mail and Internet use. The volunteers agreed to read their e-mail a minimum of twice per week and allow their Internet usage time to be monitored. Data were collected before and after the discussion period from FERN-D physicians and comparison physicians (including area physicians both members of FERN or not.)
Intervention - The specific objectives of the intervention were: 1) to increase awareness, understanding and use of evidence-based research and guidelines in clinical practice, and 2) increase use of Internet-based resources to support CME for family physicians.
Five clinical cases were prepared for the FERN-D group to support discussion of clinical issues that would influence change in knowledge and practice. These cases involved prevention topics that were selected on the basis of committee review of the Canadian Task Force on the Periodic Health Exam10 recommendations. Project staff, with assistance of family physician experts, prepared case background material, including summaries of other evidence-based guidelines and other related resources. (See Figure 1 for a list of the cases that were developed) The principal investigator moderated all the prevention cases. Each posted case followed the same format. On the first day a case scenario was posted with a couple of starter questions, followed in 2-3 days with more information, guidelines, and/or clarifications along with further questions posted by the moderator. At the end of each two-week case a summary of key points and important issues was posted. Process evaluation indicated that participants logged on an average of 4.3 times per week for an average total time of 3 hours and 10 minutes.
Ethical approval of study procedures was obtained from The University of Western Ontario Review Board for Health Sciences research Involving Human Subjects. CME credits, on a pilot basis, were obtained for FERN-D participation from the College of Family Physicians of Canada.
Data Collection - The pre-intervention mail-out survey was directed to all family physicians practicing in the area (all three groups; n = 480); the post-intervention survey was only sent to those who completed the first survey. A mail-out schedule involving three mailings (the initial mail-out, a postcard reminder, and a subsequent mail-out of a replacement survey) was followed for both surveys.11
A series of preventive practice items was included on both the pre- and post-intervention mail-out survey. Respondents indicated the proportion of patients for whom they provided a specific preventive care maneuver. For example respondents would score 1 point for providing an appropriate intervention like offering pneumococcal vaccine to all patients over 65 and 0 points for offering a chest x-ray to smokers. An overall summary score was calculated by adding scores from the 5 preventive activities discussed during the intervention, with higher scores reflecting "good" preventive practice.
Eight prevention knowledge items were included on the post-intervention survey. This set of questions was developed, and then assessed for clarity and content validity by a committee of peer family physicians. These were very specific to the cases presented and responses to these items were coded as correct or incorrect, with the percentage of respondents providing a correct response tabulated for each group.
In addition the impact of the CME intervention on the FERN-D group was sought using additional methods. This group was asked to complete an e-mail feedback survey that covered such issues as: usefulness of the component of the intervention, would they participate again, and advantages/disadvantages of this method. The FERN-D group was also invited to attend a post-intervention meeting that was conducted as a focus group to discuss their experiences in more depth. The key questions were: advantages and disadvantages of this method of CME, which did they consider good cases and why, and the role of the moderator. This meeting was audio taped, transcribed, and main themes were described.
Analysis - Assessment of the impact of the intervention involved : 1) comparison of rates of FERN-D and FERN members who indicated that they had learned new information or made practice changes as a result of their e-mail discussion group participation; 2) pre- and post-intervention comparison of preventive practice related to issues discussed for the 5 FERN-D prevention cases; and 3) post-intervention comparison of knowledge levels of FERN-D and Comparison Group physicians related to specific information from the prevention case discussions.
Results
The FERN-D group response rate to the intervention questionnaire was 85% (34/40) and to the post-survey the rate was 100% (40/40). The comparison group response rate was 50.4% (222/440) for the pre-survey and 68% (151/222) for the post survey (only mailed to responders to the first survey).
A description of the characteristics of the Educational Intervention (FERN-D) Group and Comparison Group is provided in Table 1. The groups were comparable with regard to most of the demographic and practice variables considered. The majority of the FERN-D Group and the Comparison Group practiced in an urban location, had a full-time practice with at least one other associate, and saw over 130 patients per week. While the majority of both FERN-D Group and the Comparison Group were male, there was a somewhat higher representation of female family physicians in the FERN-D Group. As well, the FERN-D Group participants had somewhat less practice experience (the mean years in practice was 3.5 years fewer in the FERN-D group), and more likely to have a full-or part-time appointment with the university than Comparison Group physicians.
With regard to the eight prevention knowledge questions included on the post-intervention questionnaire, a higher percentage in the FERN-D Group indicated accurate knowledge for seven of the items than the Comparison Group, although significant group differences were found for only two of the knowledge questions. (Item 3, Prostate CA: X2 = 17.6, df = 1, p < .001; Item 7, Folic Acid Intake: X2 = 10.5, df = 1, p < .01). See Table 2
Both the FERN-D Group and the Comparison Group improved their preventive practice scores from pre- to post-intervention. (see Table 3). A repeated measures ANOVA of preventive practice scores for both the FERN-D Group and Comparison Group revealed a significant time effect (F = 28.3, df = (1,159), p < .001), but did not identify a significant group effect.
The FERN-D Group in contrast to a subset of the Comparison Group who had experience with unmoderated medical e-mail discussions indicated greater awareness and change in practice. While 58.8% of the FERN-D Group reported becoming aware of new techniques/ research, 38.5% of the subset reported such awareness.(p>.05, n.s.) Additionally, 64.7% of the FERN-D Group reported implementing a change in their practice versus 30.8% of the subset. (p< .05, significant)
The FERN-D Group evaluated using an on-line format to do formal continuing education. The three components identified as important were: 1) end of discussion case summaries posted by the moderator, 2) the information content of case discussions, and 3) experiences and advice shared by FERN-D participants. Two advantages of the on-line format were cited (compared to interactive, face to face CME discussion groups). They were convenience (89.2%) and interaction with family physician colleagues (86.5%). Disadvantages cited by some of the FERN-D Group participants included: lack of opportunity to meet other participants face to face; and difficulties associated with on-line dialogue e.g. typing responses. A total of 97.2% of respondents would highly recommend using this type of CME to a colleague and 89.2% would agree to participate in another FERN-D group discussion.
Some themes identified from findings of the focus group included: the importance of FERN-D being organized, run and moderated by family physicians: "I think the biggest advantage of this type of CME is not constantly being told by specialists about this... I get suggestions from other family doctors who really understand what it is the problem seems to be"; convenience: "I have two kids which means that, a lot of the CME stuff that gets held, it’s difficult.. I either have to be able to drag the kid along... that’s much tougher than doing it by e-mail. I really enjoyed that."; and, on-line discussions should be case-based with cases developed on the basis of researched background material: "(the case scenarios) ..you felt they were real people....there was somebody that could walk into your office...That's what we deal with, the people." Overall, participants were supportive of ongoing development in the area of Internet-based CME for family physicians.
Discussion
The Internet-based discussion group of cases to educate family physicians on prevention issues showed higher knowledge than the comparison group and showed equal improvement in self-reported prevention practice. This case based discussion group reported they implemented significantly more changes in practice than the comparison group. The participants also reported they enjoyed this type of CME and would like to take part in a similar initiative again in the future and/or recommend this to colleagues.
The advantages cited included: convenience,use of family physician moderators, the interaction among family physicians, and the education being case-based.
These pilot results indicate enough positive influence to deserve further development and assessment. There are obvious parallels between our approach and Problem-Based small group learning. Differences include: participants are able to learn, asynchronously, at personally convenient times and, the two week time frame for each case allows time to integrate and "practice-test" new information. Theoretical work by Nowlen stressed the role of the group in adopting new information.12 Variations in the effect of interventions are not consistent across learners suggesting that progress has something to do with personal motivation and the learners perceived knowledge or performance gap with the problems presented.13 This suggests that this method needs to be targeted to particular physicians based on a needs assessment.
Limitations - The present study used a volunteer sample which may have different motivations and learning needs than a random sample. Many of our measures showed only a small change in reported behavior but many were already doing the maneuver suggesting a ceiling effect may be operating in our results. The study used self- report which may not reflect actual behavior, suggesting that direct measures (i.e. practice audit) might be recommended for future studies. Differences between the two groups due to other characteristics cannot be excluded as possible explanations for the findings. Only a randomized trial could address the latter two limitations.
It is possible that the self-selected physicians, who knew they were being studied, tried harder to get the "right answers" on our knowledge and practice change questions.
Conclusion - This pilot study shows that there is promise in using the Internet for CME if there is participant involvement. The Internet and available software is changing rapidly and the outlined approach could be repeated using new software integrating type, graphics and soon, video vignettes. The challenge is to show that these new ways of delivering CME are effective in changing physician physician behavior and, ultimately, patient outcomes. CME accrediting organizations are increasingly looking to evidence-based CME activities.
References
Davis DA, Thomson MA, Oxman AD, Haynes RB. (1995). Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995;274:1836-7.
Davis DA, Thomson MA, Oxman AD, Haynes RB. (1992). Evidence for the Effectiveness of CME: A Review of 50 Randomized Controlled Trials. JAMA 1992;258:1111-7.
Goel V, Naylor CD. Using Research and Evaluation Results in Health Services Policy Making. In, Disseminating Research/Changing Practice. Edited by: Dunn EV, Norton PG, Stewart M, Tudiver F, Bass MJ. Thousand Oaks, CA: Sage Publications 1994;199-210.
Lomas J. Diffusion, Dissemination and implementaion: Who should do what? Annals of the New York Academy of Sciences 1993;703:226-35.
Premi J, Shannon S, Hartwick K, Lamb S, Wakefield J, Williams J. Practice-Based Small-group CME. Academic Medicine 1994;69:800-02.
Armstrong D, Reyburn H, Jones R. A Study of general practitioners-reasons for changing their prescribing behaviour. BMJ 1996;312:949-52.
Parboosngh J, Lockyer J, McDougall G, Chugh U. How physicians make changes in their clinical practice: A study of physicians’ perception of factors that facilitate this process. Annals of the Royal College of Physicians and Surgeons of Canada 1984;17:429-35.
Horn KD, Sholehvar D, Nine J, Gilbertson J. Hatton C, Richert C, Becick J. Continuing medical education on the WWW: Interactive pathology case studies on the internet. Ach Pathol Lab Med 1997;121:641-44.
Connelly DP, Rich EC, Curley SP, Kelly JT. Knowledge resource preferences of family physicians. Journal of Family Practice 1990;30:353-59.
The Canadian Task Force on the Periodic Health Examination.(1994). The Guide to Clinical Preventive Health Care. Canada Communications Group- Publishing, Ottawa, Ontario.
Dillman DA. Mail and Telephone Surveys: The total design method. New York: John Wiley & Sons, 1978.
Nowlen PM. A New Approach to Continuing education for Business and the Professions. New York: Macmillan, 1988.
Fox RD, Mazmanian PE, Putnam RW. Changing and Learning in the Lives of Physicians. New York: Praeger, 1989.
Acknowledgements
This project was funded by OHCEN- the Ontario Health Care Evaluation Network. The authors would like to thank Ann Grindrod, Leslie Meredith, Sibi Samivel and Karen Cooper of the Thames Valley Family Practice Research Unit for their work on this project. We would also like to acknowledge the contribution of the late Dr. Martin Bass in initiating this project.
Marshall JN, Brett PJ, Steward MA, Østbye T. Using e-mail based continuing medical education for family physicians – Can it work? Med Educ Online [serial online] 1999;4,5. Availiable from URL http://www.Med-Ed-Online.org
Correspondence to:
Dr. J. Neil Marshall,
Thames Valley Family Practice Research Unit
Suite 245
100 Collip Circle
UWO ResearchPark
London, Ont. N6G 4X8
Phone (519)858-5028; fax(519)858-5029
e-mail marshall@julian.uwo.ca
