Measuring Strength of Motivation for Medical School

Marja GH Nieuwhof*‡, Olle ThJ ten Cate, Paul Oosterveld*, Marc BM Soethout

*University Medical Center Utrecht, School of Medical Sciences
VU University Medical Center Amsterdam, Department of Public and Occupational health.
deceased

Abstract - Purpose. Students vary in their strength of motivation to start and pursue medical training. This study was conducted to investigate the psychometric properties of a Strength of Motivation for Medical School (SMMS) questionnaire.
Method. The questionnaire was designed using an iterative method. The instrument was applied to medical students (N= 296) at the start of medical school and to potential applicants (N= 147). The stability of the concept over a six month’s time and associations with other motivation measures were studied. A separate group of potential applicants and their parents (N= 169) were asked to validate the items of the questionnaire.
Results. Cronbach’s alpha reliability of .79 was found. Test-retest reliability of SMMS-scores with a six months interval was .71. Little to no association with specific dimensions of motivation was found, except for a negative correlation with ‘ambivalence towards studying’. SMMS-scores were associated with potential applicants’ plans to apply for medical school (Spearman’s rho .65) and differentially with potential applicants’ and their parents’ judgements of item validities (.13 to .57).
Conclusions. The SMMS-questionnaire appears to be a reliable and valid instrument to measure strength of motivation for medical training in students who have just entered medical school. It may be used to evaluate the validity of selection procedures and to identify associated variables that could be used in selection procedures.

     Motivation, the “why” of all human behavior, has been set forth in a comprehensive theory by Maslow.1 According to this theory, motivation directs behavior, in order to meet basic needs on a hierarchical set of seven levels: physiological needs, need for safety, need for love and belonging, need for esteem, need for self-actualization, need for knowing and understanding and aesthetic needs. In educational settings, the strength of this motivation is often translated into the amount of time and energy people are willing to invest in learning.2,3

      Medical students are naturally motivated to become a doctor. This motivation may be generated by a need for esteem, for self-actualization or for knowledge and understanding. Its strength is in balance with whatever energy the student is willing to invest, or sacrifices the student is willing to make to meet these needs. From the beginning of their medical career, students must decide on this balance. Even without a realistic image of required investments and sacrifices, applicants for medical school have perceptions that guide their decision to start a medical career.

     If the strength of motivation for medical training could be measured as a valid construct and this construct would relate to time and energy, invested in studying and to academic achievement, it would be a worthwhile construct of help to advise students and it would be helpful in admission procedures.

     Admission procedures are frequently debated in the medical education literature, because of their unsatisfactory internal and predictive validity, specifically when it comes to non-cognitive attributes.4 In selecting medical students, admission committees typically try to make decisions based on one hand on GPAs and knowledge tests, and on the other hand on application letters, interviews, aptitude tests and references. In the latter group, committees naturally want to find proof of motivation for medical training. However, there are serious doubts about the reliability and validity of non-cognitive admission criteria such as interviews.5,6 Ranking of applicants based on interviews does not predict their success in medical school, and more in general research on the effect of non-cognitive predictors of study success is controversial.7,8

This may be caused by a lack of distinction between the quality of the motivation, i.e. a dominant reason to choose for medical school, and the strength of the motivation. In most admissions procedures, the quality of motivation and motives draws most attention. Medical schools prefer students with genuine, intrinsic interest in medicine or altruistic care-directed motives above students who aim to gain personal status or high income. Committees must weigh quite different motives, inferred from letters and interviews, and then rank applicants. It would be useful to have an instrument to measure just what sacrifices students would be willing to make to start and pursue medical school, no matter what motives they have, and to be able to rank students on this quantitative dimension.

     Clearly, one cannot simply ask applicants to state their strength of motivation, because socially desirable answers will be produced. But strength of motivation for medical school, measured in students who have already been selected and may be related to both specific biographical variables in the past that do not allow cheating (e.g., work experience in health care, high school papers on health care issues), and to academic success. Such an instrument could be used to postulate a motivation-mediated relationship between biography and success in school. Biographical variables could in turn be used in selection procedures, and strength of motivation might then play more valid role in selection procedures.

     We could not locate studies on strength of motivation as a one-dimensional concept, independent of the nature of the motives. Therefore we developed a student self-report Strength of Motivation for Medical School (SMMS) questionnaire. We defined strength of motivation for medical school as the applicant’s or student’s readiness to start and continue medical training regardless sacrifices, setbacks, misfortune or disappointing perspectives (Appendix I). The aim of our study was to investigate the reliability, internal validity and item properties of the SMMS scores. We were interested whether strength of motivation would change over time and whether it would indeed be un-associated with any specific quality of motivation. Validity would be supported if (a) in pre-medical students a determination to start medical school would be associated with SMMS scores, (b) medical students who have invested more time and energy to enter medical school show higher SMMS scores and (c) non-medical students would judge that the items would indeed be able to measure strength of motivation for medical school. These hypotheses were tested.

Methods

     Construction of the Questionnaire - The SMMS-questionnaire was constructed by the authors, combining medical, educational and methodological backgrounds, with help of a senior medical student. In an iterative procedure a large number of items were generated, discussed, rephrased, and reduced to a questionnaire of 20 statements. All statements were accompanied by a five point response scale with fixed scale anchors ranging from disagree to agree, describing situations realistic to applicants or medical students. Each item is constructed to measure the readiness to start and/or continue medical training against some sacrifice, e.g., time, money, energy. Eleven items are indicative, i.e., having a positive relation with motivation (e.g., ‘I would still choose medicine if even that would mean studying in a foreign country in a language that I have not mastered now’); nine are counter indicative, i.e. having a negative relation with motivation (e.g., ‘I would quit studying if I would be 95% certain that I could never become the specialist of my choice‘).

      Samples - The following groups participated in this survey (Table 1):

  1. At the University Medical Center Utrecht (UMCU) and the VU University Medical Center (VUmc), 405 first year medical students were asked to fill out an SMMS questionnaire in their first week of education (September 2001). Ninety of the UMCU students were invited for a repeated measurement after six months (March 2002).
  2. Potential applicants visiting a UMCU information and publicity meeting (March 2002).
  3. For the prototypicality oriented validation procedure a third group, consisting of visitors to a similar information meeting others than those of Group 2, but including parents of potential applicants. These respondents did not complete the SMMS, but judged the prototypicality of the items (March 2002).

      Additional Data - Part of the UMCU-students were invited to complete the motivation scale of the Inventory Learning Style (ILS).9 This scale consists of 20 items investigating the kind of motives for academic study including four motivation subscales: (i) vocational orientation, (ii) personal interest, (iii) self-test orientation and (iv) certificate orientation. Five items are included to investigate (v) ambivalence on study choice.
The potential applicants, Group 2, were also asked to indicate their determination to apply for medical school (‘certainly’ - ‘perhaps’ - ‘most likely not’ - ‘surely not’).

      Validation Procedures - Validity was examined within the framework of the prototypicality approach.10,11 Some behaviors, reflected in questionnaire items, are more prototypical of a trait category than others; these are supposed to be more valid than others. Within the prototypicality approach item selection and validation is done by asking relevant others (peers or expert judges) to assess the prototypicality of the constructed items for the studied trait.

      Potential applicants and their parents (Group 3) were invited to judge the strength of motivation expressed in the items of the SMMS. These groups are assumed to have clear views on motivation to start medical school from a student perspective. To simplify their task, some were asked to assess the indicative items and others the counter indicative items. The judges were given the following instruction. ‘Presume a beginning medical student makes these statements. Please judge for each statement the strength of the motivation for medical school in the student who makes the statement’. A five point scale ranging from ‘moderate motivation’ to ‘strong motivation’ was used to collect the answers in the indicative items and a scale ranging from ‘no motivation’ to ‘moderate motivation’ in the counter indicative items.

     Statistical Analyses - Item quality was assessed by means of the item-rest correlations and the loading on the first principle component. Items with either an item-rest correlation below 0.20 in a standard reliability analysis or a factor loading lower than 0.30 were removed from the questionnaire. The unrotated principal component solution was used, as only in that solution the variance explained by the first component is maximal.

      The strength of motivation of each respondent was calculated as the sum of the item scores (the scores of the counter-indicative items were reversed). For respondents with only one or two missing values, a mean substitution imputation method was used, thus extrapolating 0.3% of all item scores.

     For the prototypical validation procedure the item ranking of medical school students (Group 1) was compared with the ranking made by the potential applicant and parent judges (Group 3). In both groups the ranking of the items was determined by using the mean scores. The agreement between the judges was assessed by means of the interrater reliability. The rankings made by these two groups were compared using Spearman’s rho.

Results

     Samples - Table 1 reports the sizes of the different samples used. The response rates were high. 73%
of the student invited to complete the SMMS did so, and 81 out of the 90 invited participated in the retest. Nearly all invited to participate during the information meetings did so.

     Item selection and reliability - Four items had to be removed from the questionnaire as either the item rest correlation or the factor loading was poor. The final SMMS questionnaire contained 16 items (Appendix I). Cronbach’s alpha of this 16 item SMMS was 0.79. The retest reliability was established in Group 1b, the correlation between the first and second administrations was 0.71.

     The mean SMMS-score decreased slightly between the first and second administration (M1= 57.4; M2= 54.4; SD= 8.5; t = 4.11; df = 80; P < .00). The test-retest correlation however shows that the ranking of the students with respect to motivation is high.

Construct validity - Scores on the “Ambivalence towards studying” scale of the Inventory of Learning Styles were gathered for the UMCU of Group 1c. This scale measures the uncertainty of one’s choice for higher education training. The Pearson correlation with the SMMS-score was negative, as expected (Table 2). The correlations of the SMMS with the ILS types of motives were not significant.

      UMCU students, admitted via a qualitative selection procedure, showed higher SMMS scores (N=17, mean 60.4, sd 6.6) than those admitted with a lottery procedure (N=156, mean 56.7, sd 8.1). However, this difference did not reach significance.

     Most Group 2 respondents (N=144) answered the additional question whether they plan to apply for medical school. A positive correlation between their determination to apply for medical school and the SMMS score was found.

     Prototypicality judgment validity - Table 3 presents the results of the prototypicality validation. The judgments of prototypicality were performed separately for the indicative and counter indicative items. The inter rater reliabilities are intermediate for the indicative items and very high for the counter indicative items (Table 3). The correlation of the judges’ mean item scores and the students’ mean item scores shows concordance between evaluation of motivational strength of items and self-evaluation of motivation.

     According to the panels of judges the nine indicative items express a moderate to strong motivation. The counter indicative items show larger differences in strength of motivation expressed mostly due to item 11 in which quite a low motivation is expressed (potential applicants: 1.70; parents: 1.90).

     The correlation between mean item scores from the SMMS administration and the item rankings made by the judges is moderate to strong, except for the indicative items as judged by potential applicants panel (Spearman’s rho = .13). Significance was not reached for any of these correlations, due to the small number of items (7 and 9 respectively).

Discussion

The 16-item version of the Strength-of-Motivation-for-Medical-School questionnaire proves to be a useful and reliable instrument to measure the motivation of a respondent to start or continue medical training. All items, except one, express moderate to strong motivation, as judged by potential applicants and their parents. This implies that SMMS scores will discriminate best between persons with moderate to strong motivation. These are the students we expect to apply for medical school: those with low motivation will most likely not apply.

The SMMS-questionnaire was not constructed to be a (single) criterion for selecting medical students, but for evaluative purposes; the reliability of the questionnaire is sufficiently high for this purpose (Cronbach’s alpha = .79).12

Strength of motivation, as expressed in the items of the SMMS-questionnaire, apparently agrees with the intuitive opinions of possible applicants and their parents (Spearman’s rho 0.13 - 0.57); due to the small number of items considered (5-7), correlations were not significant.

     Surprisingly, the reliabilities of the indicative items in both the applicants and the parent judgments were lower than the reliabilities in the counter indicative items. The small variance between the indicative items indicates that the panel members found rating the items differentially to be difficult, which in turn may explain the lower interrater reliability. Panel members, instructed to think of a starting medical student, may have found positively stated items all naturally applicable and negatively stated items more thought provoking. The high judgment scores on the indicative items may reveal some ceiling effect in these items, leaving less room for discrimination and consequently leading to lower reliability.

     To obtain a better understanding of the psychological concept, underlying the answers to the SMMS-questionnaire, several validation investigations were done. A strong correlation was found between the motivation score and the determination to apply for medical school. Applicants who most certainly will apply appear to have a stronger motivation. There may be a relationship between strength of motivation and the willingness to go through a selection procedure, although we could not prove a significant relation. However, application of the SMMS questionnaire in first year medical students at the University of Amsterdam showed significantly higher scores in students who had gone through a selection procedure than those who had not (P < 0.001).13

     Although the SMMS-scores slightly decreased during the first year of medical school, strength of motivation appears to be a stable characteristic: respondents with a stronger motivation at the start of medical school show a stronger motivation after six months. Note that the observed correlation is influenced by the reliability of the measurements. Correcting for this attenuation effect would show a correlation approaching 1.00.

      As expected, the strength of motivation is not associated with one type of motivation: little correlation is found between the SMMS-scores and the four motives of the Inventory of Learning Styles. A negative correlation with ambivalence toward studying confirms our expectation that ambivalent students show low motivation.

     In summary, the SMMS-questionnaire appears to be a reliable and valid instrument for measuring strength of motivation for medical training. The questionnaire has not been used to directly select students, and will probably not be feasible as a selection instrument, as student may give socially desirable answers if they suspect that selection decisions depend on these answers. The instrument has however proved valuable for other purposes, such as post-hoc validation of existing selection procedures after decisions have been made.13 It also offers the opportunity to identify other, biographic variables that are associated with strength of motivation for medical school which could be used in selection procedures. Also, the instrument could be used to correlate strength of motivation with learning process variables and academic success.

Acknowledgements

     We thank Margje Havermans for her assistance with the construction of the questionnaire and Yvonne Steinert from McGill University for her valuable suggestions with the translation of the SMMS questionnaire.

     This research was executed with a funding of the Dutch State Department of Education, Culture and Science.

References

  1. Maslow AH. Motivation and personality. New York: Harper and Row, 1970
  2. Bradford D, Brown AL, Cocking R. (eds). How people learn - brain mind, experience and school. National Academy Press, Washington D.C., 2000.
  3. Schultz PA. Goals as the transaction point between motivation and cognition. In: Pintrich PR, Brown DR, Weinstein LE (ed). Student motivation, cognition and learning. Essays in honor of Wilbert J McKeachie. Hillsdale, New Jersey: Lawrence Erlbaum Associates, publishers; 1994
  4. Norman GR. The morality of medical admissions. Advances in Health Sciences Education 2002:9:79-82
  5. Elam CL, Johnson MMS, Wiese HJ, Studts JL, Rosenbaum M. Admission interview reports: a content analysis of interviewers comments. Academic Medicine 1994;69:S63-5.
  6. Kulantunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teaching and Learning in Medicine 2002;14(1):34-42.
  7. Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Advances in Health Sciences Education 2001;6:159-75.
  8. McGaghie WC. Qualitative variables in medical school admission. Academic Medicine 1990;65:S145-9.
  9. Rijswijk F en Vermunt JDHM. Skillful studying. Study approach, counselling book for Open University students Part 1: Inventory of Learning Styles. [in Dutch]. Heerlen/Tilburg: Open Universiteit/Katholieke Universiteit Brabant; 1987.
  10. Oosterveld P. Questionnaire design methods. Nijmegen: Berkhout; 1996
  11. Broughton R. A prototype strategy for construction of personality scales. J Personality and Soc Psych. 1984;47:1334-46
  12. Nunnally JC. Psychometric Theory. New York: McGraw-Hill Book Company; 1978.
  13. Van der Ende J, Hulsman RL, Mourer I, Boonman JH, Griffioen FMM. Evaluation of selection of medical students at AMC University of Amsterdam [in Dutch]. Amsterdam, Academic Medical Center, March 2004.

1This questionnaire has been used in Dutch and was translated for publication purposes

Reference

Nieuwhof MGH, ten Cate OThJ, Oosterveld P, Soethout MBM. Measuring strength of motivation for medical school. Med Educ Online [serial online] 2004;9:16. Available from http://www.med-ed-online.org

Correspondence

Prof dr Th.J. ten Cate,
Onderwijsinstituut UMC Utrecht,
PO box # 85060,
3508 AB Utrecht,
The Netherlands
Telephone: +31.30.2538349/2338
Fax: +31.30.2538200
E-mail: t.j.tencate@med.uu.nl

 


Medical Education Online Editor@Med-Ed-Online.org