Ambulatory Care Skills: Do Residents Feel Prepared?


Denise E Bonds, MD, MPH*†, Josyf C Mychaleckyj, MA, D Phil*‡§, Raquel Watkins, MD*, Shana Palla, MS‡, Pam Extrom*

*Department of Internal Medicine, Section on General Internal Medicine
†Department of Public Health Sciences, Section on Behavioral Medicine and Health Services Research
‡Department of Public Health Sciences, Section on Biostatistics
§Department of Physiology and Pharmacology
Wake Forest University, School of Medicine

Abstract: Objective: To determine resident comfort and skill in performing ambulatory care skills.
Methods: Descriptive survey of common ambulatory care skills administered to internal medicine faculty and residents at one academic medical center. Respondents were asked to rate their ability to perform 12 physical exam skills and 6 procedures, and their comfort in performing 7 types of counseling, and obtaining 6 types of patient history (4 point Likert scale for each). Self-rated ability or comfort was compared by gender, status (year of residency, faculty), and future predicted frequency of use of the skill.
Results: Residents reported high ability levels for physical exam skills common to both the ambulatory and hospital setting. Fewer felt able to perform musculoskeletal, neurologic or eye exams easily alone. Procedures generally received low ability ratings. Similarly, residents’ comfort in performing common outpatient counseling was also low. More residents reported feeling very comfortable in obtaining history from patients. We found little variation by gender, year of training, or predicted frequency of use.
Conclusion: Self-reported ability and comfort for many common ambulatory care skills is low. Further evaluation of this finding in other training programs is warranted.


     Over the last 20 years medicine has increasingly shifted from hospital-based practice to the outpatient setting.1-3 As a reflection of this change in the practice of medicine, the Accreditation Council of Graduate Medical Education guidelines for Internal Medicine now includes specific guidelines for training in outpatient skills.4 The most recent guidelines state that at least one-third of residents’ training must occur in the ambulatory setting.4 The American Board of Internal Medicine lists such outpatient procedures as arthrocentesis of the knee joint, and breast and pelvic examination as part of the core competencies for becoming certified in Internal Medicine.5 Previous research has demonstrated that outpatient procedures such as joint injections, sigmoidoscopy, pap smears and wet mounts are an integral portion of many internists practice.6 Recent studies have found that graduating Internal Medicine residents and the faculty who teach them feel uncomfortable performing and teaching a variety of office-based procedures.7 Competence in outpatient medical care, however, is more than the ability to perform specific procedures. History taking, counseling, and physical examination are also integral parts of the outpatient visit. Few studies have comprehensively assessed ambulatory care skills. Our objective was to assess resident and faculty perceived comfort and skill in physical exam, history taking, counseling, and procedural skills performed in the outpatient setting, and to determine if this self assessment varied by gender or level of training.

Methods

     Subjects: We surveyed the Internal Medicine residents and general medicine faculty at an academic medical center composed of an 850-bed hospital that delivers inpatient and outpatient care, and community-based clinics. All residents had one or two half-day continuity clinics at a hospital-based outpatient clinic, or a community clinic located three miles from the hospital. Faculty physicians have primary care clinics at either a separate hospital-based clinic or the community-based clinic. Faculty physicians had two to eight half days of their own primary care continuity clinic in addition to precepting in the resident clinic. The survey was mailed to all residents and faculty members two-thirds of the way through the residency year. Non-respondents received two reminder letters and one reminder phone call.

     Ambulatory Curriculum: At the time of the survey, the curriculum in ambulatory medicine consisted of one ½ day continuity clinic for first year residents each week and two half day clinics for second and third year residents. This experience was supplemented with further supervised ambulatory rotations during the first year in the faculty clinic (one month consisting of seven half day sessions each week plus didactic conferences in communication) and in the acute care clinic (one month consisting of seven half day sessions plus didactic conferences in evidence based medicine). Second and third year residents did an additional month in the acute care clinic each year. A series of weekly didactic lectures on various topics of ambulatory care medicine were also given throughout the year. At the time of the survey, there was not a formalized list of topics. Finally, residents could supplement their ambulatory experience with elective rotations that were self-styled and included options in dermatology, women’s health, and geriatrics.

     Survey: A five-page survey was devised as a quality assurance tool to assess internal medicine residents’ ambulatory care skills. Items for inclusion were based primarily on the program requirements of the Accreditation Council on Graduate Medical Education Internal Medicine residency review committee. We included skills that residents were expected to have competence in, such as the heart and lung examinations, as well as skills residents do not generally receive instruction on, but are listed in the Residency Review Committee program requirements as possible additional procedural skills (endometrial biopsy). Face validity was determined through review by faculty of the Internal Medicine Residency program. The survey was pilot tested on four chief residents and corrections made based on their feedback. The survey was divided into five sections: physical examination skills, procedural skills, history taking skills, counseling skills, and demographic questions. Twelve different organ systems were included in the physical examination section (lung; abdomen; heart; breast; ear, nose and throat, male genitourinary; female genitourinary; neurologic; back; shoulder; and knee). A four-point Likert scale used in previously published residency self-assessments8,9 was used to assess ability (perform easily alone, perform with difficulty alone, perform with assistance and cannot perform). A similar scale was used for the procedural skills section. Proficiency at seven different procedures was assessed (pap smears, wet mount/KOH prep, joint injections of the knee and shoulder, punch biopsy, flexible sigmoidoscopy, and endometrial biopsy).

     History taking and counseling sections were also assessed using a Likert scale to reflect the residents’ comfort in obtaining the information or performing the counseling. A four-point scale was used: very comfortable, somewhat comfortable, uncomfortable, and very uncomfortable. Six areas of history taking were assessed: social history, illicit drug use, medication compliance, sleep, sexual activity, and domestic violence. Seven areas of counseling were assessed: smoking cessation, living wills/advanced directives, alcohol abuse, sleep hygiene, impotence, contraceptive options and preconception counseling.

     To determine if residents’ self assessment was influenced by their predicted future use of a skill, we also asked them to predict how frequently they would use each skill in their future practice. Three options were provided for each skill: frequent, defined as at least once per week; occasionally; and infrequent, defined as less than once per year. The final section of the survey consisted of demographic information and career goals.

     Statistical Analysis: Total self-reported physician responses were computed for the four ordinal Likert categories of skill performance and predicted future frequency of use, and tabulated against physician characteristics of gender and status (faculty versus resident; year of residency). Descriptive statistics were derived from this full data table. Where the skill or frequency distribution was strongly biased against extreme categories (8 or less responses), these were merged into the next lowest or highest category respectively. From 32 skills, 28 retained more than one category with significant totals (9+). The relatively simple study design did not warrant multinomial modeling, so skill performances with 3+ residual categories were collapsed into dichotomous categories of able to perform (or comfortable) versus sum of complement categories. Additive main effects logistic regression models were constructed for each of the 28 skills with dichotomous skill as the response variable, and gender, status (faculty versus resident; year of residency), and frequency of use, as predictor categorical variables. Stepwise sub-models were assessed using the Akaike Information Criterion (AIC)10 and analysis of deviance with partial t-tests, assumed approximate chi-squared distribution, and significance level of 0.05. S-Plus version 6.0 (Insightful Corp., Seattle, WA) was used for all logistic regression modeling.

     All specific 2x2 association analyses were performed using Fisher exact tests in Intercooled Stata version 7.0 for Windows (Stata Corp., College Station, TX), with significance criterion p<0.05 (two-sided). Microsoft Excel 97 (Microsoft Corp., Redmond, WA) was used to generate the bar charts in Figure 1 and 2.

Results

     Seventy-two out of 79 residents and 19 out of 22 faculty members completed the survey for a response rate of 91% for the resident physicians and 86% for faculty physicians. Thirty-two percent of the residents were female. The majority (54%) of the residents indicated that they planned to pursue future subspecialty training while 28% planned to practice as primary care physicians. The remainder (18%) planned to practice as a hospitalist, enter a general medicine fellowship, or declined to answer the question. When asked where they planned to practice, 44% indicated they planned to seek positions in a community larger than 50,000 individuals but smaller than 200,000, 25% in a community greater than 200,000 while the remainder were split between a town less than 50,000 or other. Nearly half (49%) of the residents stated they enjoyed their outpatient continuity clinic either always or almost always, with the remainder stating they sometimes enjoyed (43%). Very few (7%) indicated that they never, or almost never, enjoyed their clinic.

     Residents reported high ability levels for common physical examination skills such as abdomen (99%), heart (97%), and lung (93% perform easily alone) (see Figure 1). Fewer residents felt able to perform breast exams (82%), ear, nose and throat (79%), genital exams (female genitourinary 68%, male genitourinary 74%,) or neurologic (69%) examinations. Respondents indicated the highest degree of difficulty in performing musculoskeletal (back 64%, knee 39%, shoulder 28%) and eye examinations (17%). Pap smears (88% easily perform) and wet mounts (63% easily perform) were the only ambulatory care procedure that most residents felt comfortable performing. Injections of either knee or shoulder received low ability ratings (33% and 11%) and most resident felt unable to perform punch biopsy, flexible sigmoidoscopy or endometrial biopsies.

     Residents generally felt less comfortable at providing counseling to patients than in obtaining medical histories (see Figure 2). Only 72% of residents surveyed felt very comfortable in counseling a patient to stop smoking. Even fewer felt very comfortable in providing counseling on advance directives and living wills (60%), alcohol abuse (57%), or proper sleep habits (52%). Counseling related to reproduction and sexual function received the lowest comfort levels with only 28% of residents feeling very comfortable on counseling patients about impotence, 28% on contraception, and 20% on preconception issues. When surveyed about history taking skills, most residents felt very comfortable in areas such as obtaining a social history (96%), use of illicit drugs (86%), and medication compliance (86%). Most felt comfortable in asking about sleep habits (75%) and taking a sexual activity history (67%) but fewer than half felt very comfortable in obtaining a domestic violence history (38%). We also compared self-reported skill level between residents (see Table 1 and 2). In general, skill levels that received low ability ratings by first year residents also received low ratings by second and third year residents. Two procedure skills were exceptions to this. Both knee and shoulder injections had more third year than first year residents report an ability to perform easily alone than other categories combined (knee 61% versus 4%, p<0.001; shoulder 23% versus 0%, p<0.05). Some improvement was seen in the counseling skills of impotence and sleep hygiene. Forty-eight percent of third year residents felt very comfortable counseling patients on impotence versus 24% of first year residents (p<0.05). Similarly, 74% of third year residents felt very comfortable in counseling patients on good sleep hygiene but only 40% of first year residents (p<0.05). History taking skills showed no variation by year of residency.

 

    We did find several differences in the self-rated ability of attending physicians to perform physical exam or ambulatory procedures compared to residents (resident percentages represent combined residents results). More faculty physicians than residents felt able to perform male (100% versus 67%, p< 0.01) and female genital (95% versus 68%, p< 0.05) examinations easily. Similarly, more faculty physicians reported being able to perform back (89% versus 64%, p< 0.05) and knee (68% versus 39%, p< 0.05) examinations easily. Markedly more faculty physicians than residents reported being able to perform an eye examination easily (74% versus 17%, p< 0.001). Procedural skills showed less variation with only knee injections having a significant increase in faculty skill level (knee: 68% faculty versus 33% residents, p< 0.01). None of the counseling or history taking skills showed significant variation between residents and faculty (see Table 2).


     We also examined differences in response by gender. More women reported an ability to perform a breast exam (women 96% versus men 75%, p< 0.05), female genital exam (87% versus 59%, p< 0.05), and Pap smear (100% versus 81%, p< 0.05) easily alone. Similarly, more women stated they were very comfortable in providing preconception counseling (35% versus 13%, p< 0.05). However, other gender specific skill such as wet mounts did not show this same trend nor did non-gender specific skills.

     Predicted future frequency was also examined. Eighty percent of those who planned to do neurology exams frequently in their future practice rated their skill level at the highest rating, able to perform easily alone, compared to only 52% of the occasional or infrequent users. Similar differences were seen for female genital urinary exams with 82% of frequent users rating their skill level as perform easily alone compared to 52% of occasional or infrequent users; pap smears 97% of frequent users compared to 76% of occasional or infrequent users; and flexible sigmoidoscopy 35% versus 5%. Domestic violence was the only history skill that showed a similar trend with 56% of frequent future users rating skill at perform easily alone level compared to 23% of the occasional or infrequent users. None of the counseling skills showed this association.

     The final statistical analysis we performed was logistic regression modeling. We found that frequent future use was the most consistent predictor of a high skill rating in many of the physical exam skills and procedures skills but not in the history taking and counseling skills. Female gender had fewer predictive associations in these models (breast, female GU, male GU, Pap smears, and flexible sigmoidoscopy). Faculty was most predictive of high skill ratings in those physical exam skills less routinely performed in the inpatient setting (for example, genital exams, musculoskeletal exams). Male gender was not predictive of a high rating in any of the skills. There were few skills that skill level could be predicted by year of residency training.

Discussion

     We found low levels of ability to perform common physical exam and procedure skills, and low degrees of comfort in performing counseling often done in the ambulatory setting. While most residents and faculty physicians felt able to perform physical exam skills such as lung, heart and abdomen exams, less than half felt able to perform knee, shoulder or eye exams easily alone. Similarly, while most residents surveyed felt comfortable in performing a Pap smear, fewer than half felt they could perform a knee or shoulder injection. Residents generally felt more comfortable in obtaining history from patients, although many reported lower comfort levels when asking about domestic violence. However, counseling patients was a skill that many residents reported having some degree of discomfort, particularly in areas related to sexuality, or reproductive health.

     Our survey was administered to internists in training. We expected to find lower ratings in first year residents and our results confirm this. However, we found little increase as residents advanced in their training. Skills that received low self-assessments by first year residents such as counseling on impotence or contraception, received a similarly low rating by second and third year residents. There are several possibilities that may account for the lack of increase with resident year. Our sample is a single cross-sectional snapshot of internists’ abilities and may include residency classes especially lacking in self-confidence. We feel this is unlikely as no incremental improvement was seen between any two years. Additionally, the pass rate for the 2001 American Board of Internal Medicine was 93% for this residency program, above the national average. The low self-reported rating may represent an under assessment of actual ability to perform the skill. We did not objectively measure the skill in question but previous research has found that internal medicine physicians do poorly on objective tests of cardiac and pulmonary physical examinations.11,12 This sample of physicians rated their skill level very high in both of these areas, suggesting a self-assessment bias toward over assessment of actual ability. Additionally, if physicians were underestimating their skill level, we would expect similarly low scores across all skills. Instead, we found high self-rated ability and comfort levels for physical exam and history taking skills that are common to both inpatient and outpatient medicine, for example heart exams and taking a social history, and low skill levels in those skills more specific to ambulatory care.

     The most likely cause of the selective low skill level is true inability or discomfort on the part of the resident. Many of the residents indicated they would pursue further subspecialty training, thus their interest in learning ambulatory care skills may be low. However, when we compared predicted future use of a skill with self-rated skill level we found correlation only for physical exam and procedure skills. Another cause may be the lack of training provided for the skill in question. Supporting this is the similarly low ratings seen among the faculty physicians. Although faculty physicians generally rated their physical exam skills higher than residents, their ability and comfort in procedure and counseling skills was generally low. In our survey, only 58% of faculty physicians indicated they felt very comfortable counseling on smoking cessation, a skill that receives an “A” rating by the US Preventative Task Force13, and only 68% in counseling on alcohol abuse, a skill specifically mentioned by the Residency Review Committee as desirable for resident training. 4 This low comfort level may translate into limited teaching of these skills to residents.

     Although this survey is limited to one academic medical center, similar findings have been obtained in other studies. Wickstrom et al surveyed 331 general internists from nine different residency programs and found low levels of confidence in faculty physicians for teaching common ambulatory procedures such as knee injections and punch biopsy.7 Coodley et al surveyed over 300 internists about their residency training and current practice patterns for office gynecology. They found that most respondents received little training during residency on the management of common gynecology problems, but reported encountering these problems frequently during their clinical practice.14 Similarly, a questionnaire on preconception care found that few of the internists surveyed possessed the knowledge needed to provide recommended preconception counseling.15

     There are limitations to this study. We surveyed residents and faculty from only one academic medical center. Our response was very high, 90%, and included residents and faculty in both the primary care and categorical tract. However, it is possible that a survey of a different residency program may result in different assessments. This is a cross-sectional survey and the particular group of residents surveyed may not have been representative. This survey was not inclusive of all ambulatory care skills and the results may represent weakness of the program surveyed. We did, however, include a variety of skills that are either recommended by the residency review committee or the American Board of Internal Medicine.4,5 When we examined the results, we found little evidence of specific trends in self-rated skills.

     Ambulatory care is an increasingly important part of internal medicine. A recent analysis of the ecology of medical care estimated that every month 217 out of 1000 individuals in the United States visit a physician in an outpatient setting, and 113 of these visits are to a primary care physician’s office. In comparison, only 8 of those 1000 individuals are hospitalized.16 Internists are a major provider of outpatient medical care.17 Our study found that internists in training have lower levels of confidence or comfort in their ability to perform many ambulatory care skills. Other research has supported this finding.7,14,15 Future research surveying a nationwide sample of residents and faculty on a comprehensive set of ambulatory care skills would help delineate the problem. If this finding is supported by other studies, further examination of the residency curriculum and faculty development in areas of weakness may be warranted.

References

  1. Kimball HR, Petersdorf RG. Back to the future for internal medicine? Am J Med 1998; 104:315-6.
  2. Association of American Medical Colleges. AAMC policy on the generalist physician. Acad Med 1993; 68:1-6.
  3. American College of Physicians Task Force on Physician Supply. The role of the future general internist defined. Ann Intern Med 1994; 121:616-622.
  4. Program Requirements for Residency Education in Internal Medicine: American College Graduate Medical Education, 2001.
  5. American Board of Internal Medicine. Policies and Procedures 2000. Vol. 2001, 2000.
  6. Wigton RS, Nicolas JA, Blank LL. Procedural skills of the general internist. A survey of 2500 physicians. Ann Intern Med 1989; 111:1023-34.
  7. Wickstrom GC, Kelley DK, Keyserling TC, et al. Confidence of academic general internists and family physicians to teach ambulatory procedures. J Gen Intern Med 2000; 15:353-60.
  8. Woolliscroft JO, Palchik NS, Dielman TE, Stross JK. Self-evaluation by house officers in a primary care training program. J Med Educ 1985; 60:840-6.
  9. Mullen PB, Blitz SG, Stross JK. Faculty expectations and primary care resident's perceptions concerning residents' growth in competence at one medical school. Acad Med 1992; 67:113-117.
  10. Venables W, Ripley B. Modern Applied Statistics with S-PLUS. New York: Springer-Verlag, 1999.
  11. Mangione S, Nieman L. Cardiac auscultatory skills of internal medicine and family practice trainees: a comparison of diagnostic proficiency. JAMA 1997; 278:717-722.
  12. Mangione S, Nieman L. Pulmonary auscultatory skills during training in internal medicine and family practice. Am J Resp Crit Care Med 1999; 159:1119-11124.
  13. Guide to Clinical Preventive Services. In: Force USPST, ed. Alexandria, Virginia: International Medical Publishing, 1996.
  14. Coodley GO, Elliot DL, Goldberg L. Internal medicine training in ambulatory gynecology. J Gen Intern Med 1992; 7:636-9.
  15. Conway T, Hu TC, Mason E, Mueller C. Are primary care residents adequately prepared to care for women of reproductive age? Fam Plann Perspect 1995; 27:66-70.
  16. Green LA, Fryer GE, Jr., Yawn BP, Lanier D, Dovey SM. The ecology of medical care revisited. N Engl J Med 2001; 344:2021-5.
  17. Care Survey: 1998 Summary. Advance data from vital and health statistics. Hyattsville, Maryland: National Center for Health Statistics, 2000.

Reference

Bonds DE, Mychaleckyj JC, Watkins R, Palla S, Extrom P. Ambulatory care skills: Do residents feel prepared? Med Educ Online [serial online] 2002;7:7. Available from URL http://www.med-ed-online.org

Correspondence

Denise E. Bonds, MD, MPH
Section on General Internal Medicine, Department of Internal Medicine
Wake Forest University, School of Medicine
Medical Center Boulevard
Winston-Salem, North Carolina 27157
Telephone: (336) 716-6012
Fax: (336) 716-7359
Email: dbonds@wfubmc.edu

 

Medical Education Online Editor@Med-Ed-Online.org