;  
Google
 
 		   ;

Teaching Physicians Procedural Skills at a National Professional Meeting

Patrick C. Alguire, MD


Director of Education and Career Development
American College of Physicians
Philadelphia, PA

Abstract: Background: Practicing physicians often wish to improve their procedural skills but have limited educational opportunities to do so.
Description: To summarize the effects of two procedural workshops on participants’ confidence, proficiency, and practice patterns.
Evaluation: Following completion of a skin biopsy or arthrocentesis workshop, participants completed a post-course and an 8-month follow up evaluation. Recipients of this training rated it highly and reported that following training they performed more procedures, referred less, and noted an increase in their confidence that was still evident eight months after the workshop.
Conclusion: Skin biopsy and arthrocentesis/joint injection skills can be taught to practicing physicians in a workshop setting at national professional meetings. Key Words: clinical competence; internal medicine; teaching; educational measurement

          Many patients, especially those in ambulatory settings, expect their physician to perform needed procedures. Patients benefit from the continuity, convenience, and in certain situations, decreased cost when their personal physician performs a procedure. Yet, surveys of general internists indicate that many do not feel confident in performing common ambulatory procedures, citing inadequate training.1-4 To further address the procedural training needs for practicing internists, a telephone survey of a random sample of 101 members of the American College of Physicians (the “College”) who attended the annual scientific meeting was conducted in 1997 to determine how the College might meet the changing educational needs of its members. The College is the professional society for internal medicine physicians and has over 80,000 general and subspecialty internal medicine members. The survey was part of regularly scheduled educational needs assessment of College members performed every 2 to 3 years to assist in the planning of educational programs, products, and services for its members. Using a 5-point Likert scale with the descriptive anchors of “little or no interest” to “very interested,” subjects were asked to rate their educational needs for a variety scientific, clinical, and medical computing topics. Comments were also solicited. Members rated their interest in learning procedural skills as highly as scientific and other clinical topics. In the area of procedural skills, 56% and 60% were “interested” or “very interested” in learning skin biopsy and arthrocentesis, respectively. Interest in learning these two procedural skills was significantly greater among generalists when compared to specialists. In 1998 the College first offered workshops in skin biopsy techniques followed by the introduction of arthrocentesis and joint injection workshops in 1999. While the skin biopsy and arthrocentesis and joint injection workshops have been offered annually since their introduction, this study was conducted in 2000.

          Training physicians in new procedures can be problematic. Barriers include: lack of trained faculty, adequate patient numbers, cost, and access to equipment.5,6 Alternatively, systematic instruction of students and residents in workshop settings is reliable and cost-effective and offers the opportunity to practice new skills in a “safe setting,” thereby enhancing skill retention and minimizing errors.5,7-11 Studies have shown that procedure workshops for students and residents improve perceived competence, operator safety, and patient outcomes.7,8,11 These successful workshops adhere to established principles of effective medical education; they address the learners’ needs, include interaction among learners with opportunities to practice the skills learned, and utilize multifaceted educational activities. It was assumed that a similar workshop format would be successful in teaching procedural skills to practicing physicians.

        The purpose of this paper is to describe and summarize the preliminary results of using an interactive workshop format to teach skin biopsy and arthrocentesis skills to practicing internists. We wished to study the impact of the workshop on the participants’ frequency of performing the procedures, referral patterns, and self-reported proficiency and confidence.

Methods

          The setting for the workshops was the American College of Physicians annual scientific meeting. This meeting attracts between 5500 to 6000 physicians for 3 days of continuing medical education activities. The majority of the sessions utilize traditional lecture formats but one area was set aside for interactive learning activities. This area is known as the “Learning Center” and was the setting for the skin biopsy and arthrocentesis and joint injection workshops. Workshop faculty members were recruited from the College membership who had previously demonstrated interest and ability in teaching these procedural skills at other meetings. Each workshop was one-hour in duration and was repeated six to eight times daily throughout the course of the three-day meeting.

          The workshop model for both the skin biopsy and arthrocentesis/joint injection training consisted of demonstration, practice, and feedback. Learners were required to observe either a live or videotaped demonstration of the procedure anytime prior to attending the one-hour workshop. During the demonstration, the faculty reviewed the rationale, cited the indications and contraindications for the procedure, listed the necessary equipment, and demonstrated each component step of the procedure. The live demonstration was 90 minutes in duration and was facilitated by projecting the performance of the procedure onto a screen to accommodate large audiences. The videotape demonstrations of the procedures, produced by the College specifically for the workshops, were 20 minutes in duration and were shown continuously to groups of 20 to 25 participants in the Learning Center next to the workshop practice stations. The 20-minute videotape contained all of the essential core content believed by the faculty to be necessary to participate in the workshop. This core content was also presented in the live demonstrations in addition to content that was not relevant to the workshop. In the skin biopsy demonstration, three skin biopsy techniques (shave, punch, and excisional) were performed on a pig’s foot model. In the arthrocentesis demonstration, anatomic landmarks were identified on a “live patient” model and the skill was demonstrated on polyurethane models of the shoulder, knee, and hip that emitted an electronic beep when a needle was properly inserted into the joint space. During the subsequent hands-on practice, learners used the same models.

          Following observation of the skill, participants came to the workshops and were divided into small learning groups with a faculty to learner ratio of approximately one to five. Upon entry into the workshop, each participant was given a “criterion checklist,” a concise listing of the sequential and essential steps necessary for satisfactory performance of the procedure. For each skill, the faculty demonstrated a component step in the skill, and the participants then performed the step. This sequence was followed until all steps in the procedure were completed. The faculty provided coaching and feedback during the entire process. At the end of the session, all participants were given a course syllabus, samples of patient education materials, evaluation and management coding information, examples of procedure notes, a list of necessary equipment, and post-procedure patient instructions. The overall goal of the workshops was to provide a comprehensive and systematic introduction to the procedure, rather than to ensure competence. Before performing the procedure on a patient, participants were advised to seek additional training under the supervision of an experienced preceptor.

          In 2000, the present study was undertaken to measure the effect of the two workshops on the participant’s confidence, proficiency, and practice patterns. At that year’s annual scientific meeting, 450 physicians participated in the skin biopsy workshop and 586 in the arthrocentesis and joint injection workshop over the course of 3.5 day meeting. Using a Likert scale where 1 = strongly disagree and 5 = strongly agree, participates rated the workshop along the following parameters: videotape demonstration was helpful, feedback from faculty was helpful, sufficient faculty contact time was given, sufficient practice time was available, and course objectives were achieved. Using a Likert scale, where 1 = not confident or not proficient and 5 = very confident or very proficient, participants were asked at the end of the workshop to rate their confidence and proficiency in performing the procedures before and immediately after the workshop, a measurement technique known as “retrospective rating” (that is, both ratings are done at one point in time). Retrospective ratings have been used to maximize consistent criteria for pre-and postintervention self-assessments.12-16 By collecting preintervention and postintervention ratings simultaneously, both ratings are based on the same conceptualizations of the skill being assessed and provides a common standard for both ratings. Retrospective pre-intervention and post intervention ratings have been shown to be more sensitive and valid as compared to traditional pre and post-intervention ratings (that is, ratings done at two different points in time) for assessing change.17

          The last question of the evaluation asked participants if they would consent to a follow up evaluation. Those who consented were mailed an eight-month, follow up survey for the workshop they attended. Participants who attended both workshops received both surveys. In the follow up survey, participants were asked to estimate the number of skin biopsy or arthrocentesis procedures performed and their referral patterns before and after the workshop. Finally, each subject was asked to rate his/her current confidence in performing specific biopsies (shave, punch, and excisional) and arthrocentesis of specific joints (knee, hip, shoulder) using a dichotomous scale (yes or no). Participants were not asked if they had received additional training since the index workshops, and patient outcomes or satisfaction was not assessed.

          Responses were coded to assist in the follow up of non-responders but were otherwise anonymous. Follow up survey data collection was completed in two months. Descriptive statistics (frequency, means, standard deviations, and percents) were used to quantify the number of procedures performed and number of patients referred. Pre and post-workshop comparisons of these data were analyzed with the Wilcoxon signed ranks test. Pre and post-workshop ratings of confidence and proficiency were reported as means and standard deviations, and the results were compared using Student’s t test for paired samples. For all comparisons, the critical significance criteria, p-value for a two-tailed test, was < 0.05.

Results

          Immediately following the Annual Session 2000 workshops, 204 of the 450 (45%) skin biopsy participants and 289 of the 586 (49%) arthrocentesis/joint injection participants completed a course evaluation. The participant mean ratings for the workshops ranged from 4.3 to 4.7 for skin biopsy and 4.1 to 4.4 for arthrocentesis. Confidence in performing a skin biopsy increased from 2.2 + 1.1 before the training to 4.2 + 0.8 immediately after. The difference between the group means was 2.0 (95% CI = 1.8 to 2.1; Student’s t = 27.9; df = 177; p < .001) Proficiency scores increased from 2.1 + 1.1 before the training to 3.9 + 0.8 immediately after. The difference between the group means was 1.8 (95% CI = 1.7 to 1.9; Student’s t = 27.1; df = 183; p < .001). Confidence in arthrocentesis/joint injection increased from 2.4 + 1.0 before the training to 4.0 + 0.7 immediately after. The difference between the group means was 1.6 (95% CI = 1.5 to 1.7; Student’s t = 31.4; df = 248; p < .001). Proficiency scores increased from 2.5 + 1.1 before the training to 3.9 + 0.8 immediately after. The difference between the group means was 1.5 (95% CI = 1.4 to 1.6; Student’s t = 27.6; df = 255; p < .001).

          The response rate for the 8-month follow up survey was 66% (135 of 204) for the skin biopsy workshop and 67% (193 of 289) for the arthrocentesis workshop. This represented 30% of the original skin biopsy participants and 33% of the original arthrocentesis/joint injection participants. Table 1 shows that at eight months respondents reported performing significantly more skin biopsies, arthrocentesis/joint injections following the workshop (skin biopsy z = 6. 4, arthrocentesis z = 4.9, joint injection z = 5.4; p < .001 for all comparisons).

As seen in Table 2, respondents also reported referring significantly fewer patients to specialists for these procedures (skin biopsy z = 6.4, arthrocentesis or joint injection z = 6.8; p < .001 for both comparisons).

Confidence varied according to the type of skin biopsy procedure performed or joint location. Eight months following the workshop, when asked if they were confident or not confident in performing the procedures, skin biopsy respondents reported the most confidence in performing a punch biopsy, and arthrocentesis respondents reported the most confidence in aspirating or injecting a knee joint. A small percentage of respondents never gained confidence in any of the procedures (Table 3).

Discussion

          Skin biopsy and arthrocentesis/joint injection skills can be taught to practicing physicians in a workshop setting at national professional meetings. Recipients of this training rated it highly and reported that following training they performed more procedures, referred less, and noted an increase in their confidence that was still evident eight months after the workshop.

          The workshops were designed along the principles of effective medical education articulated by Davis et al and others.18-21 These principles include an assessment to identify the learners’ needs as the initial step. Other steps shown to be effective in changing physician behavior include focusing on practical skills, active participation with an ability to practice what is learned, and multi-faceted and sequenced learning activities. Finally, providing information or tools to assist the practitioner in incorporating the new skill into the practice setting also contributes to change.

          The workshop adhered to many of these principles. A needs assessment survey was performed on a representative sample of learners who had attended an Annual Session. Topics were selected that were both highly rated and clinically important to practicing physicians. A sequenced, multifaceted approached was used consisting of watching a demonstration (either live or videotaped) followed by hands on practice of the skills and the use of checklists to facilitate learning and retention, and the provision of feedback and coaching by experienced faculty. In order to facilitate the incorporation of the newly learned skills into the practice, participants were provided with samples of patient education materials, evaluation and management coding information, examples of procedure notes, a list of necessary equipment, and post-procedure patient instructions.

          A few of the teaching techniques deserve special mention. The videotape demonstrations were highly rated by the learners. Videotapes can help the learner conceptualize the application of theoretical knowledge and can be reviewed in whole or in part many times until the learner becomes familiar with the content.22,23 Visual images in combination with verbal instruction have also been shown to significantly increase recall and retention.22 Criterion checklists, or reference guides, are useful because they provide a concrete referent and help focus the learner’s attention on the critical aspects of the skill.24 Criterion checklists are also useful after the training as a reminder of the essential steps in the procedure. The ability of an experienced faculty providing positive and constructive feedback appears to be a critical component in promoting change. For example, using videotape instruction without faculty contact is associated with more learning failures and a reduced likelihood of using the skill in clinical practice.25,26

          This is a preliminary report on the effect of interactive workshops on the learning of office-based procedural skills. It has a number of limitations including the low response rate of only 45% to 50% immediately after the intervention, dropping to just over 30% at 8 months. This significantly limits confidence in the results and their generalizability. The study is also limited by its reliance on self-reporting of perceptions as the major outcomes rather than objectively measuring the learners’ performance against established criteria or actual patient outcomes, including patient satisfaction. Participants may have exaggerated their estimates of procedures performed, referrals made, and perceived level of confidence pre and post-workshop, making the intervention appear more effective than it actually was. It is unknown if the participants received additional training following the workshop. If this did occur, it could have contributed to their reported increased confidence in performing the procedures. While asking participants to retrospectively rate their knowledge, skills, and attitudes pre-intervention is a validated methodology, a concurrent control group would have strengthened the study. There was a small group of participants who never gained confidence in performing the procedure. No additional data was collected on these individuals to know why they never gained confidence. Knowing this information might be helpful in designing future workshops. Overall, however, the promising results reported here suggest that more careful study of this teaching format is worthwhile.

          We conclude that a workshop model that incorporates the basic principles of adult learning, and utilizes the teaching strategies of demonstration, hands-on practice, and feedback is a feasible teaching format for practicing physicians. Additional studies will be needed to determine if this type of educational intervention results in more procedures performed and fewer referrals as well as improved patient satisfaction and outcomes.

Acknowledgments

The author gratefully acknowledge Carol Popkave for her assistance in the survey design and data collection and analysis, Dr. Gerald Arnold for data analysis, and Dr. Herbert Waxman for his thoughtful review of the manuscript.

References

  1. Wickstrom GC, Kolar MM, Keyserling TC, et al. Confidence of graduating internal medicine residents to perform ambulatory procedures. J Gen Intern Med 2000;15:361-65.
  2. Wickstrom GC, Kelley DK, Keyserling TC, et al. Confidence of academic general internists and family physicians to teach ambulatory procedures. J Gen Intern Med 2000;15:353-60.
  3. Mandel JH, Rich EC, Luxenberg MG, Spilane MT, Kern DC, Parrino TA. Preparation for practice in internal medicine: a study of ten years of residency graduates. Arch Intern Med 1988;148:853-56.
  4. Kern DC, Parrino TA, Korst DR. The lasting value of clinical skills. JAMA 1985;254:70-76.
  5. Norris TE, Cullison SW, Fihn SD. Teaching Procedural Skills. J Gen Intern Med 1997;12:S64-70.
  6. Sierpina VS, Volk RJ. Teaching outpatient procedures: most common settings, evaluation methods, and training barriers in family practice residencies. Fam Med 1998;30:421-23.
  7. Powers LR, Draeger SK. Using workshops to teach residents primary care procedures. Acad Med 1992;11:743-45.
  8. Sherertz RJ, Ely EW, Westbrook DM, et al. Education of physicians in training can decrease the risk for vascular catheter infection. Ann Intern Med 2000;132:641-48.
  9. Snell GF. A method for teaching techniques of office surgery. J Fam Prac 1978;7:987-90.
  10. Heppell J, Beauchamp G, Chollett A. Ten-year experience with a basic technical skills and perioperative management workshop for first-year residents. CJS 1995;38:27-32.
  11. Fincher RME, Pogue LN, Cowan CF. Teaching correct and safe bedside procedures. Acad Med 1991;66:396-97.
  12. Bray J, Maxwell SE, Howard GS. Methods of analysis with response shift bias. Educ Psycho Measure 1984;44:781-804.
  13. Hoogstraten J. The retrospective pretest in an educational training context. J Exper Educ 1982;50:200-04.
  14. Hoogstraten J. Influence of objective measures on self-reports in a retrospective pretest-postest design. J Exper Educ 1985;53:207-10.
  15. Howard GS, Ralph KM, Gulanick NA, Maxwell SE, Nance SW, Gerber SK. Internal validity in pretest-postest self report evaluations and a re-evaluation of retrospective pretests. Applied Psy Measure 1979;3:1-23.
  16. Sprangers M, Hoogstraten J. On delay and reassessment of retrospective preratings. J Exper Educ 1988;56:148-53.
  17. Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: a comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Eval Health Prof 1992;15:351-66.
  18. Davis DA, Thompson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education activities. JAMA 1995;274:700-05.
  19. Davis D, O'Brien MAT, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education. Do conferences, workshops, rounds and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282:867-74.
  20. Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: Guide to the evidence. JAMA 2002;288:1057-60.
  21. Cantillon P, Jones R. Does continuing medical education in general practice make a difference? BMJ 1999;318:1276-79.
  22. Dwyer FM. Strategies for improving visual learning. State College, PA:Learning Services;1978.
  23. MacKinney AA. On teaching beside diagnostic and therapeutic procedures to medical students: an annotated bibliography of audiovisual materials. J Gen Intern Med 1994;9:153-57.
  24. Yelon SL. How to use-and create-criterion checklists. Perform Instruct J 1984;April:1-4.
  25. Baldwin D, Hill P, Hanson G. Performance of psychomotor skills: a comparison of two teaching strategies. J Nurse Educ 1991;30:367-70.
  26. Ventres WB, Senf JH. Introducing a procedure using videotape instruction: the case of the lateral birth position. Fam Med 1994;26:434-36.

Reference

Alguire PC. Teaching physicians procedural skills at a national professional meeting. Med Educ Online [serial online] 2004;9:1. Available from http://www.med-ed-online.org

Correspondence

Patrick C. Alguire, MD
American College of Physicians
190 North Independence Mall West
Philadelphia PA 19106-1572

Email - palguire@acponline.org

 

Medical Education Online Editor@Med-Ed-Online.org