D.A. Penava^* and S. Stanojevic^†

     The physician-patient relationship is vital to the practice of clinical medicine. Communication is a key component of this relationship and is also the foundation of many clinical skills.¹ Good communication skills are highly correlated with patient satisfaction, compliance, perception and physician competence.^1,2 Most medical schools provide training in interpersonal skills; few, however, have established standardized evaluation procedures for communication skills.¹

     With the evaluation tools available,^3-6 setting standards for communication skills can be difficult. One of the difficulties in assessment of communication skills lies in who functions as the assessor.^{5, 7-10} Physicians act as assessors for clinical skills on standardized examinations; some have demonstrated that they may be suitable for assessing communication skills as well. ^7,8-10 Others, however, have shown a discrepancy in communication skills assessment between physicians and standardized patients (SPs).⁵ Literature reporting poor correlation between medical students self-assessment and patient assessment further supports the notion that physicians assess skills differently than patients. ^{1,4, 7}

      As its intervention, this study utilizes the established medical student run Healthy Sexuality program, in which medical students volunteer as peer counselors for adolescents in a local high school. ¹¹ The program framework is based on the Information-Motivation-Behavior skills model; it is designed with the premise that preventive behavior is a function of an individual’s information about prevention, motivation to engage in prevention, and behavioral skills for performing the specific acts involved in prevention.^12,13 The Healthy Sexuality program was designed with the goal to provide adolescents with behavioral skills that can be utilized in social situations of increasingly risky sexual behavior.

      In this study we proposed that students who voluntarily participated in the Healthy Sexuality program (gaining experience counseling adolescents) would perform better on an adolescent counseling station on the standardized year-end Objective Structured Clinical Exam (OSCE). This assessment was performed using a communication skills tool developed by the author (DP) for this purpose. In addition, we investigated to determine if student self-assessment is a reliable measurement compared with adolescent “patients”.

Methods

      Medical students at the University of Western Ontario were recruited to participate in the Healthy Sexuality program at a peer presentation during a volunteer job fair approximately three weeks after beginning medical school. Participation involves self-learning around topics of interest as well as a “core” curriculum presented in a hard copy binder for students to review. Role-play scenarios are presented and medical students rehearse these at their own discretion with peer presenters.

      The program involves presentation of relevant sexual education material and initiation of role-playing scenarios catered specifically to the needs of high school students. One public secondary school’s grade nine students participated through their health education classes. Role-playing scenarios include addressing issues of initiating conversation, perceived gender roles, peer perceptions and pressures, boundary setting, different types of intimacy or sexual activity, symptoms of STDs, being “prepared” for responsibility, use of contraception and condom purchase and application. At the end of the sessions, medical students were asked to assess their own as well as a colleague’s communication skills using the tool in Figure 1a. Adolescent participants were asked to evaluate the medical students using a modified tool (Figure 1b). Assessments were performed on a 7-point Likert scale.

      To assess inherent differences between students (Healthy Sexuality volunteers and non-volunteers), a pre-intervention survey of year one medical students was administered before volunteers for the Healthy Sexuality program were recruited. Students were asked about age, gender and any experience with counseling or communication skills they may have had before beginning medical school.

     At the year end OSCE for first year students one of the ten-minute stations was designed to evaluate the students’ ability to communicate with adolescents; students were instructed to take a history focused on the adolescent’s concerns. Students were evaluated on their history- taking ability by physician examiners; the assessment is in Figure 2a. Standardized Patients assessed communication skills for study purposes only, shown in Figure 2b.

Mann Whitney non-parametric tests and Pearson correlations were performed on the data using SPSS software. This study was funded by the Academic Enrichment Fund in the Department of Obstetrics and Gynaecology at the UWO and was approved by the UWO Ethics board.

Results

      Nineteen first year medical students (14 female and 5 male) participated in the voluntary Healthy Sexuality program. In analyzing the assessments made during the Healthy Sexuality Program, it was determined that medical students self-assessments correlated significantly with peer-assessments (Pearson Correlation; r = 0.62). However, neither peer-assessment (r = 0.03) nor self-assessment (r=0.05) correlated with assessments made by the adolescents who participated in the Healthy Sexuality Program. Medical students’ received a mean score of 6.1 (range 1.0 to 7.0) from participating adolescents.

      At the year end OSCE all first year medical students (57 male and 45 female) were rated on communication skills: physician examiner scores were recorded as a total score per student, SP scores were averaged from the 9 item 7 – point scale. Program participants did not perform better than their peers on a standardized adolescent counseling station at the final OSCE (Mann-Whitney U test; p = 0.42 and p = 0.81 for SPs and Physician Examiners respectively). Results were not influenced by students’ gender. Graphical representations of scores given by SPs (Figure 3) and Physician examiners (Figure 4) are given. The mean SP score for Healthy Sexuality participants was 6.1, (range 4.7 to 6.9). Standardized Patient assessment of non-participants had a mean score of 6.3 (range 4.1 to 7.0). Physician examiners assigned scores that ranged from 7 to 14 for participants and from 6 to 13 for non-participants. The mean physician examiner scores were 10.1 and 10.2 out of 14 for the participant group and non-participant group respectively. Correlation between physicians and SP scores was statistically significant (Pearson Correlation; r = 0.62).

Discussion

      Our communication skills assessment tool was easily incorporated into the standardized OSCE with little or no adjustment to the exam procedure already present. It is anticipated that the communication skills tool designed for this study may be useful for standardized OSCEs or clinical exposures; this is an area of further study. Given the clustering of scores observed (Figure 3), we believe its optimal use would be to identify students who fall below a set standard, and whose skills would benefit from additional training. Previous research has concluded that communication skills training can improve history-taking confidence and that students with more training asked fewer questions at examinations and were rated as more empathetic.¹⁴ Timing of such an assessment is very important; we recommend that communication skills be assessed at the end of the first or second year of undergraduate medical education in an attempt to improve skills for students who require it.

      Participation in the Healthy Sexuality program was hoped to result in improved communication with adolescents. Contrary to what was expected, participation did not result in an objectively measurable improvement in communication skills. This may have occurred because the development of communication skills is not stressed in an objective manner in this student-run program. Even without this finding however, adolescent and medical student participants in the program appreciate the experience. Objective benefit may be in medical student attitudes and the adolescents’ behavior, which is an issue to be assessed in another manner.

      This study confirmed previous results that self-assessment is not a reliable tool for measuring communication skills, as medical students’ assessment did not correlate with adolescent assessments.^4,14 Peer-assessment was not reliable either, as the assessments made by adolescents did not correlate at all with those made by peer medical students. The wide range of scores given to medical student participants from the adolescents may represent the true communication skills of the students; however, they also could be indicative of the varied sexual experiences of the adolescent participants. For example, although the program is tailored to the specific needs of the group, some areas may be perceived as being too elementary for some students, and this may be reflected in poor scores for medical student educators.

      We did find a significant correlation between SP evaluation of communication skills and physician examiner assessment of history taking skills (Pearson Correlation; r = 0.67). This finding contradicts results of previous studies.⁵ It is somewhat difficult to interpret a true correlation given the differences in assessment tools between SPs (communication skills; Figure 2b) and physician examiners (history taking; Figure 2a). In fact, upon review of the physician’s checklist (Figure 2a), the points given likely represent an indirect assessment of communication skills that were rated by SPs (Figure 2b). Further work in the area of who should assess communication skills is needed to delineate these differences. Our research is now focused on objective assessment and validation of our communication skills tool. It is hoped to then be used early in medical education to identify those students who require extra training.

References