Observational Validation of a Model of Ambulatory Teaching*
Traditionally, medical education has been inpatient-oriented. As social, technological, and economic pressures have forced more patient care into ambulatory settings, increased attention has been given to medical education in those settings.1 The need for ambulatory education and the concerns about the costs and logistical problems of carrying out teaching programs in ambulatory settings have been widely discussed.2-4 Now there is a need for more specific information regarding the nature of ambulatory teaching and the evaluation of its effectiveness.5,6 Irby, in an extensive review of research on ambulatory teaching, characterizes education in the ambulatory setting as variable, unpredictable, and lacking in continuity.7 The specific goals of the teaching experience can greatly influence what is taught, where it is taught, and how it is taught.8,9 A better definition and understanding of the nature of ambulatory education and the teaching process that occurs within these settings may assist medical educators improve the education that occurs there. If we can develop a more consistent model of ambulatory teaching with specific teaching behaviors, it would aid in the assessment of teaching effectiveness and in the development of more targeted faculty development programs. A recent study identified a model of ambulatory teaching in a longitudinal program in primary care for third-year medical students. The model identified a shifting role for faculty as students’ experience proved they warranted greater responsibility. Teaching roles moved from Modeling to Supervision to Consultation.10 The model was developed from encounter log data and interviews of faculty and students. Prislin et. al.11 found similar documentation of increasing patient care responsibility of students in a longitudinal experience. Another recent study by Henkelman et al, using direct observation of teaching, identified three components of ambulatory teaching: 1) Orientation to the patient — the learner presenting a case to the teacher; 2) Confirmation of findings — the teacher reexamines the patient or in other ways confirms the learner's findings; 3) Closure — the teacher gives information and direction regarding patient management.12 The current study attempted to validate the three-stage, progressive model of teaching suggested by Biddle et. al.10 by sampling preceptor/student encounters throughout the course of a longitudinal ambulatory clerkship. Methods The Clinical Campus at Binghamton, a branch campus of the State University of New York Health Science Center at Syracuse, has offered a yearlong, longitudinal, primary care clerkship since 1979. All third-year Clinical Campus students participate in the program, during which they see patients one half-day per week in the office of a primary care physician. Each student works in the same office with the same faculty preceptor for the entire year. In the academic year 1995-1996, thirty primary care physicians (28 Family Medicine and 2 Internal Medicine) served as preceptors for the longitudinal primary care clerkship. Seven (23%) of these preceptors (6 Family Medicine and 1 Internal Medicine) volunteered to participate in the study. There were no statistically significant differences between the volunteers and the non-volunteers on measures of student assessment of teaching effectiveness or amount of teaching experience. The overall teaching effectiveness ratings from students on a five-point scale (5 = excellent and 1 = poor) were 4.71 for the volunteer subjects and 4.61 for the non-volunteer group (p<.61). The volunteer subjects’ average length of time teaching in the program was 13.4 years compared to 9.4 years for the non-volunteering preceptors (p<.08). The teaching activities of the preceptors were assessed by direct observation. The preceptor/student encounters were observed in patient examining rooms, physicians’ offices, hallways, and other areas where preceptors and students interacted regarding patient care or other educational topics. To ensure observation of all preceptor/student encounters, the observer followed the students through all of their activities during each observation session. No video or audio tapes of the teaching sessions were made due to the need to move throughout the office setting and to observe the wishes of the preceptors. Each preceptor was observed once for a half day session during each third (beginning, middle, end) of the longitudinal experience. To maximize the possibility of identifying differences in teaching strategies over the course of the ten-month experience, the periods observed were defined as one-month blocks, with three and one-half months separating each observed period. The first observation block began with the third week of the clerkship to ensure that all orientation to the clerkship and specific sites was completed. The first observation occurred during week 3 to 6 of the clerkship. The second (middle) observations occurred in weeks 20 to 24 and the last (end) occurred in weeks 36-39. An ambulatory clinical teaching observation instrument was designed for this study to capture all possible teaching behaviors related to the three stages of the model reported by Biddle et al.10 The form was based on one devised for observation of inpatient teaching.13 It was checked against the list of teaching behaviors identified in a recent study in an ambulatory setting,12 and behaviors from that study not already included on our form were added to complete the form for the current study. Thirty teaching behaviors were included on the observation form. In addition, the categories "No Teaching" and "No Contact" were added. "No Teaching" was defined as the preceptor and student in the same room or location with no specific teaching activity observed. "No Contact" was defined as the student and preceptor in different locations during the observed time interval. The form was also subdivided to allow for recording the location where teaching occurred. The location headings used for the form were the examining room, preceptor’s office, and other. The observation form was reviewed for face validity by a panel of four faculty involved in primary care training. The instrument was also piloted with two preceptors who taught in the course, but were not part of the volunteer sample used in the study. During the piloting the observer recorded then reviewed the findings with the observed preceptor immediately following the observation session to see if the recorded behaviors matched the perceptions of the preceptors in order to establish some level of content validity. In order to analyze their contribution to the teaching model, the thirty teaching behaviors were grouped into five categories: Modeling, Supervision, Validation, Consultation, and General Teaching (Table 1). Table 1
Observations were done by one of the authors (WBB). The observer recorded the teaching behavior occurring at the end of each 30-second interval including the category No Teaching. An audible tone was used to indicate the 30-second observation intervals. The audible tone was recorded on a 30-second loop of audio tape and was heard through an earphone from a small pocketsize tape recorder. The observer also recorded the number of patients seen in each half-day session and the beginning and ending times of each session. Data for individual teaching behaviors were merged into the five categories identified above. Comparisons were made on each of the categories, plus the No Contact data for the three observations using Friedman Two-Way ANOVA. Wilcoxan Matched Pairs Signed Ranks tests were used to follow-up on significant results from the Friedman ANOVA.14 Nonparametric analyses were used due to the small sample size. An alpha level of .02 was used for the Wilcoxan tests to adjust for the multiple (3) comparisons made on each variable. The frequency of teaching behaviors was also summarized for the five categories by location. Comparisons were made on each of the five categories for the three locations using Friedman Two-Way ANOVA followed by Wilcoxan Matched Pairs Signed Ranks tests as done for the category by time analyses. Results The frequency of occurrence of the teaching behaviors, as summarized within the five categories over the three observations is presented in Figure 1. Statistically significant differences (p<.005) were identified for Modeling and Validation. The other categories showed no significant change over the three observation periods. The average for Modeling fell from 147 in Period 1 to 63 in Period 2 and 40 in Period 3. The difference from Periods 1 to 3 was significant (p<.02), while the drop from Periods 1 to 2 followed the expected trend but did not reach significance (p<.03). For Validation, the increase from 9 in Period 1 to 67 and 58 respectively for Periods 2 and 3 was statistically significant (p<.02). The frequency of No Contact intervals (30-second blocks) increased over the course of the observations (Period 1=90.29; Period 2=182.43; Period 3=187.71). The change from Period 1 to Period 2 was significant (p<.02). The predominant location for teaching was the examining room. For the Modeling, Supervision, and Validation categories, there were statistically significant differences between the amount of teaching that occurred in the exam room as compared to any other location (Table 2). For General Teaching and Consultation the preceptor’s office was the most frequent location, but the differences did not reach statistically significant levels in either case. Table 2 Mean number of 30-second intervals in which teaching behaior was displayed by primary care preceptors
* p<.01
Discussion The results of this study suggest that teaching behaviors do change over the course of a longitudinal clerkship. The decrease in Modeling coupled with the increase in Validation lends support to the proposed model of ambulatory teaching. There was no direct confirmation of the proposed Supervision stage. Also, the Consultation stage seems to rely more heavily on validation activities than on consultative behaviors. Alternatively, the Supervision stage may have occurred earlier than anticipated and was largely missed by the schedule of observations used. The fact that a significant increase in validation behavior occurred as early as the second observation is consistent with this explanation of a faster transition in the model. More frequent observation of the teachers is required to test this alternate explanation. Another factor that may be affecting the third stage of the model, Consultation, is the leveling off or slight decline in total teaching behaviors from Period 2 to Period 3. This can be accounted for partially by the increasing time students spend with patients before interacting with the teacher, a recognized part of the teaching model. In other words, as students spend more time with patients on their own there is a decrease in the opportunity for direct teaching and thus a decline in total teaching behavior. This is partially supported by the No Contact data that showed a substantial increase from the first to second observations. The findings on location of teaching are consistent with the results that show Modeling and Validation as the most frequent types of teaching behaviors. These behaviors rely largely on the presence of patients and therefore are most likely to occur in the examining room. The Consultation category contains the behaviors most likely to occur in other settings such as the preceptor’s office. In this study, the actual frequency of consultation behaviors was very low, thus contributing little to the occurrence of teaching in these other locations. The level of the learner in this study may have influenced the frequency of consultation behaviors. All learners were third-year medical students; many other studies of ambulatory teaching have included first - through third-year residents. Preceptors working with residents may engage in consultative teaching more than preceptors in this study did. The current study lends support to a multi-stage model of ambulatory teaching, particularly to the early stages of Modeling and Validation. Some caution needs to accompany this conclusion, given the limitations of the study. The study involved a relatively small sample of volunteer preceptors. Although no statistically significant differences could be identified between the volunteers and other preceptors in the course, there is concern that volunteers are more likely to be confident about their teaching and more comfortable in the role of preceptor. This may not be a major concern for identifying a model for successful teaching, but it needs to be kept in mind as one looks at generalizing to groups of less experienced teachers. Another potential limitation on the generalizability is the fact that a single observer collected all data for this study. There needs to be additional work done to show that the observation instrument produces similar findings in other settings with other observers performing the assessments. Additional research with larger samples and with teachers from a variety of programs would also increase the generalizability of the findings. Block as well as longitudinal programs should be included to detect if the same progression of teaching occurs in both formats. Most importantly, more frequent observations should be made in future research to increase the likelihood of observing possible short-term changes in the teaching process. It should also be pointed out that the model of teaching being studied was developed from data on a longitudinal primary care clerkship for third year medical students. Other experiences with different goals and for learners at other levels may require different teaching strategies. Therefore, additional observation of teaching needs to be conducted. Future studies should include observations in a variety of programs and with multiple levels of learners to fully understand the range of teaching models that are appropriate in ambulatory education. If additional support for the multi-stage model of ambulatory teaching can be found, then more specific efforts in faculty development and assessment of teaching effectiveness can be implemented. Faculty development programs could begin with a reemphasis on Modeling, with particular attention given to specific behaviors found to be associated with that stage of teaching. Examples would include directing students to watch certain aspects of the doctor/patient interaction and learning to debrief students on what they saw and what they can learn from what they observed. If, as in this study, there is a limited amount of teaching at the consultative or problem-solving level, then attention can be given to teaching skills that encourage this level of learning. In addition, preceptors could be taught to change the goals and process of their teaching to the next level or stage as students progress through the model. The characteristics from the multistage model and the teaching behaviors related to each stage ,as identified in this study, could be used to develop criteria for assessing preceptor performance. This information could be used for feedback and other faculty improvement strategies. To build better faculty development and assessment procedures, more data on teaching models and the specific teaching behaviors associated with them will be needed. This study only provides a narrow glimpse at what one proposed model may look like. Further validation of this and other models in a variety of settings and with a broad spectrum of faculty will be required to move ahead in a systematic fashion References
Biddle WB, Riesenberg LA, Erney S, Siska K. Observational validation of a model of ambulatory teaching. Med Educ Online [serial online] 1998;4:8. Available from URL http://www.Med-Ed-Online.org. Dr. Biddle is Associate Dean for Academic Affairs, SUNY Health Science Center at Syracuse, Clinical Campus at Binghamton. He can be reached via electronic mail at BIDDLEB@VAX.CS.HSCSYR.EDU |
|||||||||||||||||||||||||