An Assessment of the Impact of Multimedia, Technology-Based Learning Tools on the Cardiac Auscultation Skills of Third-Year Medical Students
Dario M. Torre, MD, MPH*‡, Kurt
J. Pfeifer, MD*, Geoffrey C. Lamb, MD*,
Cardiac auscultation is a valuable and cost-effective tool in the diagnosis of patients with heart disease.1,2 However, despite widespread acceptance that proficiency in cardiac auscultation is an important component of the physical exam, previous research has shown a deficiency in this bedside skill among medical students and other post-graduate trainees.3,4,5
Ideally, cardiac auscultation should be taught at the bedside by experienced clinicians who observe, coach, review and reflect with their students as each (teacher and pupil) examines actual patients with abnormal cardiac findings. However, today’s clinical environment creates barriers to this traditional educational approach. Significant erosions in the amount of time available for clinical teachers to fulfill their educational responsibilities, combined with decreasing lengths of stay for sicker, hospitalized patients, have created an educational environment where students and teachers find it difficult to effectively learn, practice and perfect their cardiac auscultation skills .6,7,8
Fortunately, the recent growth of technology has created new opportunities to practice cardiac auscultation skills. Multimedia-based teaching programs, combining the reproduction of cardiac sounds and interactive case based teaching, have shown promising results in enhancing learners’ auscultation skills.9,10,11 In particular, several studies of web-based learning materials have reported high levels of student satisfaction, improved learner confidence, and demonstrable gains in knowledge.12,13,14 Despite the potential advantages of a web-based cardiac auscultation curriculum, the unique educational contribution of web-based learning to enhance clinical skills, like cardiac auscultation, requires additional study.15 Furthermore, the emergence of personal digital assistants (PDAs) has created additional opportunities in medical education by making it possible to: document each learner’s unique clinical experiences: identify high quality teaching activities, and; bring teaching resources directly to the point of learning.16,17,18 We hypothesized that combining the interactive nature of web-based learning with the bedside portability of a PDA would achieve the ideal of bringing this type of instruction back to the patient’s bedside and enhance the ability of medical students to learn cardiac auscultation more effectively as compared to simply listening to computer generated sounds in a lecture format.
Study Participants - Thirty-three individuals, forming two study populations (medical students and residents), served as participants based on their random assignment to medicine rotations during the study period. Third year medical students (M3) rotating on the required two-month clerkship in internal medicine at the Medical College of Wisconsin (MCW) in Milwaukee, Wisconsin formed the medical student study population. As is typical with many required medicine rotations, students complete a one-month hospital-based rotation and a one-month ambulatory clinic block as part of their required core clerkship experience in internal medicine. Ten first year residents (PGY-1) and 5-second year internal medicine residents (PGY-2) completing one of their required inpatient months during the study period served as the resident population.
Attrition due to failure to complete the posttest or withdrawal from the rotation, reduced the final number of participants to 29: thirteen IM residents (9 PGY-1 and 4 PGY-2) and 16 M3 students. Attrition rates were similar in the two groups (2/15 [14%] for the resident group and 2/18 [12%] for the medical student group.
Study design and setting - This was a retrospective study designed to assess the cardiac auscultation skills of a group of third year medical (M3) students with access to a lecture and two self-directed technology-based learning tools compared to a group of more senior level trainees who were only exposed to the lecture. The intervention group was comprised of all 18 third year medical students who attended a cardiac auscultation lecture and had used at least one technology-based learning tool (web or PDA) during their rotation. Students were included if they self-reported that they had used the web site at least once per week and/or had recorded at least two heart sounds/murmurs onto their PDAs (provided by the Dean’s Office to all third year medical students). The comparison group for this study consisted of a convenience sample of 13 internal medicine (IM) residents at the same institution who were attendees at an academic half-day skills program, a structured part of the curriculum for all IM residents at MCW. As part of their clerkship curriculum and clinical skills evaluation, M3 students were required to pass a criterion-referenced cardiac auscultation test given at the end of the clerkship. All M3 students had equivalent access to computer-based resources at our affiliated teaching hospitals and ambulatory training sites. As the use of web and PDA-based learning tools is an integral part of our clerkship curriculum to teach students cardiac auscultation and improve the quality of students’ education, this project was exempted from by review according to MCW IRB regulations.
Educational Intervention - Our cardiac auscultation curriculum, targeted to M3 students, was comprised of three different technology-based learning tools: a PDA-based heart sounds/murmur form19 seen in Figure 1; a web-based cardiac auscultation program20 seen in Figure 2; and a cardiac auscultation lecture that featured computer-generated heart sounds. Internal medicine residents, as part of their clinical skills program, received only the simulator-based cardiac auscultation lecture. Residents had neither access to the cardiac auscultation web site nor to the PDA-based forms, as both were password protected. All M3 students were asked to download a locally developed PDA-based heart sounds/murmur form at the beginning of their two-month clerkship in internal medicine. This self-directed learning tool included the main clinical characteristics of six systolic murmurs (aortic stenosis, mitral and tricuspid regurgitation, mitral valve prolapse, hypertrophic obstructive cardiomyopathy [HOCM] and innocent murmur), two diastolic murmurs (mitral stenosis and aortic regurgitation), two extra heart sounds (a third heart sound [S3] and a fourth heart sounds [S4]) and fixed and physiologic splitting of second heart sound [S2]. For each heart sound/murmur, the PDA-based form listed the best area of auscultation, timing, change with maneuvers and associated physical exam findings. After evaluating patients with cardiac abnormalities, students were asked to record their findings on the PDA form using a simple touch screen response (Yes/No/Don’t know). PDA forms were downloaded onto a centralized database during the clerkship’s weekly Core Curriculum program.
The second element of the curriculum was a web-based cardiac auscultation site that included information on the same murmurs and heart sounds that were included on the PDA-based form. In addition, the web site also included information about physiologic and fixed splitting of S2. The cardiac auscultation website was designed using “Front Page®” web-authoring soft ware. Simulated cardiac murmurs and heart sounds were provided by Cardionics® (Houston, Texas). For each murmur/heart sound, the website included a diagram of the chest with a “hot spot” that identified the best area of auscultation and a hypertext link to an embedded multi-media sound file. Each murmur/heart sound was accompanied by a table that summarized timing, change with maneuvers and/or position, and associated cardiac physical exam findings.
The third element of the curriculum consisted of a one-hour cardiac auscultation lecture that featured computer-generated heart sounds and murmurs. The same faculty member facilitated this session each time it was presented. M3 students received this lecture midway through their two-month medicine clerkship. IM residents received the cardiac auscultation lecture approximately halfway through the academic year, as part of a regularly scheduled clinical skills-based curriculum. The content and format of the lecture was identical for both groups of learners. During these sessions, students and residents used infrared stethophones to listen to heart sounds as they were taught how to identify the key features of cardiac sounds/murmurs based on timing, pitch, best area of auscultation, and response to diagnostic maneuvers. Each lecture presentation focused on the same heart sounds and murmurs that were featured on the PDA-based form and the web-based learning program.
The selection of specific cardiac physical findings for inclusion on the multi-media tools was based on information derived from the literature, including the results of a national survey that asked residency program directors to rate the cardiac physical exam findings that they felt were most important to teach.21
Evaluation - At the end of the two-month internal medicine clerkship, we evaluated the cardiac auscultation skills of M3 students via a twelve-item performance-based exam that required students to correctly identify various heart sounds and murmurs that were generated by CardioSimR (Houston, Texas), a computerized heart sound simulator. Medical residents completed the same performance-based exam approximately one week after receiving the simulator-based cardiac auscultation lecture. Development of the cardiac auscultation exam was based on an assessment blueprint that was organized around the key features of the heart sounds and murmurs included in the multimedia-based learning tools and the simulator-based cardiac auscultation lecture. Test items for the cardiac auscultation exam were selected based on reports from previous literature21 and further developed by a consensus opinion of the MCW medicine clerkship directors and internal medicine faculty members.
During the examination, all participants (students and residents) were told the chest area where the sound was heard and, for selected sounds, any radiation or change with maneuvers. After listening to each tested sound for a period of two minutes, all examinees were required to choose a single-best answer from a multiple-choice listing of possible auscultatory findings. No heart sounds or murmurs were repeated beyond the allotted time. To avoid the introduction of bias, the test was administrated separately to medical students and residents by the same faculty member. All test items were piloted for clarity and reliability among a group of thirty M3 students who completed their required medicine clerkship earlier in the academic year. The Chronbach’s alpha reliability coefficient for the cardiac auscultation skills test was 0.92.
All M3 students (n=16) who participated in this study also completed a 10-item questionnaire that was designed to assess their frequency of use and level of satisfaction with the multi-media technology-based learning tools described above. As the use of web-sites and PDAs are required elements of the clerkship, problems with access or functionality are typically immediately reported to the clerkship director. During the study period, no problems with functionality or access for either the PDA or web-based applications were reported.
Data analysis - Mean scores for the student and resident groups were computed with standard deviations. The t-test for unpaired data was used to compare the performance of each group on the 12-item auscultation exam. Chi-square tests were used to compare the scores of M3 students and medical residents with respect to each auscultatory finding.
The mean score of correct responses on the cardiac auscultation exam for M3 students was 9.6 (range 1-12 with a standard deviation [SD] of 1.6) compared to a mean score of 7.2 (range 1-12 with SD 2.3) for medical residents. Our finding that M3 students had a higher overall mean score of correctly identifying heart sounds/murmurs compared to medical residents was statistically significant (p <. 005). Overall, students correctly identified 80% of the computer-simulated heart sounds/murmurs while the medical residents accurately detected 60% of the same cardiac findings. Table 1 demonstrates that, when each test item was analyzed separately, M3 students had significantly higher identification rates than medical residents for the following heart sounds: physiologic splitting of the second heart sound (S2), fixed splitting of S2, third heart sounds and fourth heart sounds (S3 and S4 gallops) with p < 0.005 for each of these findings. Resident performance appeared to be about equal to that of M3 students in correctly identifying the majority of the study-specific heart murmurs, although medical residents had slightly higher identification rates for the murmurs of mitral valve prolapse and HOCM. The results for these latter two findings, however, did not reach the level of statistical significance.
All M3 students reported that they used at least one technology-based learning tool to improve their cardiac auscultation skills during the time of this study. Almost all of the M3 students (15/16, 94%) reported using the cardiac auscultation web site, while 75 % (12/16) used the PDA-based murmur forms and 69 % reported using both.
Of the fifteen students who reported using the cardiac auscultation web site, all reported access rates averaging once per week (range 1-3 times per week). Eighty percent (80%) of these users indicated that the site contributed positively to developing their cardiac auscultation skills and 90 % felt that the site promoted self-directed learning.
The twelve M3 students who utilized the PDA-based murmur form recorded a total of 63 heart sounds/murmurs, with an average of four heart sounds/murmurs per student (range 2 - 11). All students who used the PDA-based form reported hearing an extra heart sound, including ten who noted an S3 gallop and six who auscultated a fourth heart sound. All twelve students recorded entries for aortic stenosis and mitral regurgitation while eight students reported hearing diastolic murmurs, including aortic insufficiency (n=7) and mitral stenosis (n=1). Three students recorded entries for tricuspid insufficiency and one student reported examining a patient with HOCM. Only one student entered a description of an innocent murmur on their PDA-based murmur form.
Our report suggests that third year medical (M3) students who used a combination of technology-based learning tools may be able to achieve a greater level of proficiency in cardiac auscultation than previously published baseline measures and higher rates of performance than local, but more senior level, trainees who were exposed to only one of these modalities. The use and availability of two different technology-based learning tools may have created opportunities for practice and repetition of such skills both in a low risk learning environment and at the bedside, ultimately creating an incentive for students to practice cardiac auscultation with live patients.
Our report confirms the results of previous studies that have shown reasonable improvement in cardiac auscultation skills following the use of computer-based instructional programs.9,11,22,23 In addition, our PDA-based murmur form allowed students to bring technology to the bedside, thereby facilitating key learning processes of cueing, reinforcement and documentation while directly interacting with patients. Point of learning use of PDAs may foster students’ practice of key clinical skills in other areas of the physical exam.
Previous literature has described learning variations among individuals in the way they approach learning tasks according to their personality traits and learning environment, and the complex manner in which learners perceive and process information they are attempting to learn.24,25 As reported in this study, the use of learning tools that utilize multiple modalities and encourage self-directed learning may facilitate the diverse learning needs of our students and trainees.
Several limitations of our study deserve consideration. First, the lack of baseline cardiac auscultation skills performance measures, the small sample size and the lack of a true control group precluded the opportunity to compare the relative impact of various instructional interventions by group. However, Mangione reported that for selected auscultatory findings, residents and students had at least comparable levels of cardiac auscultation proficiency.3 Secondly, the need to pass the test may have provided medical students with stronger motivations to learn compared to residents. Therefore, particularly for those goal-oriented students, such motivation may have affected participation, learning, and possibly performance. However, we should consider that there are other types of learners such as learning-oriented and activity-oriented learners, for whom the motivations are created by the activity itself or by learning for its own sake. Third, we did not to track the amount of actual time that students spent on the technology-based tools, which could have provided more detailed information about the students’ use of such tools.
Two other study limitations merit discussion. First, the cardiac auscultation skills of medical students and residents were assessed by measuring their ability to identify computer-generated heart sounds/murmurs rather than by examining real patients with actual cardiac abnormalities. The “clean findings” generated by the heart sound simulator, played at a constant speed and volume and free of background noise and other distractions, may have facilitated recognition of cardiac events. Although it is possible that proficiency in recognizing computer-generated sounds may not reflect the real-world ability of students and residents to correctly identify heart sounds/murmurs at the bedside, previous research has shown that use of a cardiac simulator does improve student performance not only on a simulation-based auscultation exam, but also on a patient-based skills test 26
Second, long-term retention of cardiac auscultation skills cannot be assessed by the results of our study. However, because the technology-based tools that we described can be used at the learner’s own discretion and pace, students should have more opportunities for self-directed learning and skills-based practice. These enhanced opportunities may result in greater durability and retention of the ability to correctly identify heart sounds and murmurs. Finally, our cohorts consisted of third year medical students and internal medicine residents from a single institution. Results from other institutions and other groups of learners may vary depending on the learner’s baseline level of proficiency and previous exposure to teaching/learning activities that focused on developing competency in this part of the physical exam.
Multimedia based simulations are not intended to replace the patient as the focus for teaching cardiac auscultation. Bedside instruction that pairs a seasoned clinician and a memorable patient remains the ideal method of teaching this important skill. Cardiac auscultation is best learned via hands-on coaching, patient-centered practice and constant repetition, but the time available for this type of clinical education is limited in today’s fast-paced health care environment. The use of multiple technology-based modalities appears to be an effective and efficient strategy for teaching and reinforcing cardiac auscultation skills. These types of technology-based teaching interventions may be of particularly great value when brought to the point of learning and integrated, in a real-time fashion, with meaningful clinical scenarios. Furthermore, the use of such tools may foster the development of self-directed learning by allowing students and trainees to select a modality that best matches their preferred learning style. Future research should focus on defining the generalizabilty of our findings to other medical schools and residency programs, and measuring the specific impact of technology-based learning tools on patient care and clinical outcomes.
Torre D, Pfeifer KJ, Lamb GC, Walters MP, Sebastian JL, Simpson DE.An assessment of the impact of multimedia technology-based learning tools on the cardiac auscultation skills of third-year medical students. Med Educ Online [serial online] 2004;9:22. Available http://www.med-ed-online.org.
Dario M Torre,MD,MPH