Wireless Handheld Computers in the Preclinical Undergraduate Curriculum

Anand C. Ganger, BS* and Matt Jackson, PhD†

*CampusMobility®
528 S. State Street, Suite 531
Ann Arbor MI 48104

†Dept Immunology and Microbiology
Wayne State University School of Medicine
540 E. Canfield Ave.
Detroit MI 48201

Abstract: This report presents the results of a pilot project using wireless PDAs as teaching tools in an undergraduate medical curriculum. This technology was used to foster a transition from a passive to an interactive learning environment in the classroom and provided a solution for the implementation of computer-based exams for a large class. Wayne State Medical School recently provided model e570 Toshiba PocketPCs® (personal digital assistants or PDAs), network interface cards, and application software developed by CampusMobility® to 20 sophomore medical students. The pilot group of preclinical students used the PDAs to access web-based course content, for communication, scheduling, to participate in interactive teaching sessions, and to complete course evaluations. Another part of this pilot has been to utilize the PDAs for computer-based exams in a wireless environment. Server authentication that restricted access during the exams and a proctoring console to monitor and record the PDA screens will be described in this report. Results of a student satisfaction survey will be presented.

Keywords: Pocket PC, PDA, wireless exams, interactive teaching, real-time assessment

     Due to large class sizes, it would not be possible to meet the changing demands in medical education without the implementation of technology-enhanced instruction. Medical students are required to possess the knowledge and skills for basic clinical diagnoses by the completion of their sophomore year. Therefore, a focus in medical education is to promote problem-based learning using clinical vignettes and encourage independent acquisition of information from web-based resources.1 To achieve this goal, a current trend in medical education is to supplement the traditional didactic lecture experiences with small group, collaborative learning. Handheld computers have an increasing role in clinical practice for patient databases, prescription writing, and subject content. Therefore, medical school educators have a responsibility to train future physicians to become skilled using the latest technologies such as handheld computers to access current information, for communication, and patient tracking applications.2,3

     Wayne State Medical School is in the process of digitizing course content and implementing the use of wireless personal digital assistants (PDAs) into the undergraduate medical curriculum. The wireless PDAs are being used for the delivery of digitized course content, interactive classroom sessions, small group problem-solving exercises, communication, computer-based testing, real-time evaluations, and the collection of patient encounter information during clinical clerkships. A goal of these initiatives is to facilitate the transition from a passive to an interactive learning environment.

PDAs are becoming increasingly prevalent in clinical practice.4-7 Physicians and hospital administrators are finding that the handheld devices are invaluable tools for generating patient databases, prescription writing, and information retrieval. In addition to basic course management tools, the wireless devices foster the establishment of an interactive learning community comprising students, faculty, and practicing physicians.8

     In collaboration with CampusMobility®, Wayne State Medical School has embarked on an initiative to introduce state-of-the-art technology into its undergraduate medical curriculum. A pilot project that provided wireless PDAs to a group of second year medical students was initiated in the summer of 2002. Wireless access points were installed in a lecture auditorium and in the small group teaching labs used for undergraduate medical courses to accommodate this initiative. Overflow of the transmission from these locations extended the range of wireless access to other locations in the building.

     Our pilot group of students has been successfully using the devices in the interactive classroom. This report describes the technical and logistical considerations for selecting suitable hardware/software platforms to meet specific educational needs. Our experiences with the deployment of this technology in the undergraduate medical curriculum are also presented.

Implementation

      Wayne State Medical School received support in early 2002 from the Provost’s Office to purchase Toshiba model e570 PocketPCs®, wireless network interface cards, and application software. PocketPC®s were chosen over PalmPilots due to the wireless local area network (WLAN) capabilities, additional memory, expansion slots, and graphic capabilities (Table 1). PocketPC®s meet our needs for application in the undergraduate medical curriculum. We have developed interactive, web-based tutorials with digital images and instructional videos as part of the curriculum made accessible using the PDAs. Wireless PocketPC®s were also more suitable for our interactive teaching and computer-based testing applications.

      A list-serve message was distributed to the freshman class in the spring of 2002 requesting participation in the PDA pilot project. This solicitation described the scope of the project and our expectations of the participants. Thirty percent of the class responded to the invitation to participate and a lottery was used to select 20 participants from the pool of respondents. Thirty-nine percent of the respondents were women and 61% were men. This gender distribution was reflected in the 20 students selected to be the pilot group using the lottery system. Seven (35%) of the participants were women and 13 (65%) men. As a whole, this class was more evenly distributed with 53% men and 47% women indicating that the group of initial respondents was somewhat skewed toward men. There was a range of prior experience with personal computers and handheld devices amongst the pilot group participants. Thirty-three percent of the respondents had previous experience using a PalmPilot and 20% had used a PocketPC®. Fifteen percent of the pilot group had prior experience working with a PDA or laptop computer in a wireless environment.

      The pilot group was assembled the first week of classes for distribution of devices and a 1-hour orientation session. During orientation, the students were provided with a description of basic PDA functions, wireless access settings were established on each device, and dual e-mail accounts were set up. The latter function permitted the students access to their School of Medicine account as well as a separate CampusMobility® account for functions such as calendar updates, special announcements, and technical support. A portal was established for the students to access year 2 content, interactive questions, evaluation forms, surveys, and wireless exams for the Microbiology, Immunology, and Infectious Diseases course. CampusMobility® provided mobile device management to manage problems students may have encountered with non-functioning hardware and outdated software prior to wireless sessions in the lecture auditorium.

     Sixty per cent of the students required hands-on technical assistance, primarily caused by the loss of data when the PDA battery was allowed to completely drain for an extended period. In these cases, it was necessary to re-install application software from a compact flash card or by synchronization with a desktop PC. Battery life when accessing web-based course content in the wireless environment was a limiting factor. A one hour interactive teaching session or wireless exam would result in an approximately 70% loss in battery power. These technical issues have been addressed by the manufacturer since the inception of the pilot project. Extended battery capacity has been provided in newer models, preventing data loss. A high capacity battery is available for the model e740 Toshiba s that has an extended capacity approximately five times the standard battery.

      A set of survey questions was posted on the portal for on-line participation by the pilot group of students. Survey questions addressed areas that we felt would be problematic when using the wireless PDAs for exam administration. A Likert Scale was used to gauge the student satisfaction with aspects such as the authentication process for examinees, reading the questions, viewing clinical microbiology images on the small screen, and having their on-line exam activities electronically monitored. Other survey questions addressed general PDA uses such as scheduling, communication, and software preferences.

Collaborative Learning

     Our large class size of approximately 250 students limits faculty and student interaction. Therefore, as part of the pilot project, the wireless PDAs were used to introduce interactivity into the didactic presentations. A series of multiple choice questions were published on the portal for the PDA users to access during the lecture. Questions were simultaneously projected in the lecture hall for the entire class to view. The PDA group submitted a response from a multiple choice list and an audience response graph was available in real time to the lecturer who was logged onto the administrator section of website. As with the multiple choice questions, the audience response statistics were projected in the lecture hall for the entire class to view. This system permitted the lecturer to view the audience response and respond to areas of weakness in real-time. A typical session with 3 case presentations, 10 questions pertaining to the case, and discussion required approximately 20 minutes of classroom time.

     Faculty participation was a key component of the pilot project, as well as an indicator of future class-wide implementation. Forty-two percent (11/26) of the Immunology, Microbiology, and Infectious Disease teaching faculty participated in interactive teaching sessions using the wireless PDAs. The remaining faculty were either unwilling to adapt the new classroom method (7/26 or 27%) or were not asked to submit interactive questions to the portal due to their limited participation (1 contact hour) in the course (8/26 or 31%). In future implementations, all of the faculty will be asked to participate in the interactive questions, regardless of contact hours.

     Although a limited number of students were participating in the pilot phase of this initiative, we gained valuable insight regarding this technology for large scale implementation in the future. With the cooperation of the faculty, the wireless PDA-based audience response system successfully introduced interactivity, an element of surprise, and controlled chaos into the classroom. Information relating to class-wide implementation was gained by informal interviews of the participants and surveys that were designed for access using the wireless handheld devices. Results of an on-line survey revealed that PDA use among the group was approximately 1 hour per day. The School of Medicine calendar access and schedule management were the most popular non-wireless feature of the PDA (65% response) while accessing and downloading medical software was the most popular (55% response) wireless feature available to our PDA group. Examples of software preferred by the first and second year medical students was Taber's Cyclopedic Medical Dictionary, 19th Edition by D. Venes and C.L. Thomas (F.A. Davis Company) and the Lippincott Williams & Wilkins titles Anatomy Recall PDA by L.H. Blackbourne, Pathology Recall PDA by A. Chhabra, L.H. Blackbourne, and B.D. Blackbourne, and USMLE Step 1 Recall PDA: Buzzwords for the Boards by B.A. Reinheimer.

Wireless Testing

     A transition to computer-based U.S. Medical Licensure Exams occurred in 1999.9 Many medical schools lack the infrastructure for computer-based assessment of their undergraduates. We have implemented the use of PDA-based medical school exams in a wireless environment to address this limitation.

     The chief concern in migrating to WLAN access is security.10,11 Physical wires are one of the primary obstacles to attackers looking to hack their way onto a LAN. This has been relatively controlled, as it’s not likely that a stranger plugging into a network would go unchallenged, either by the network security that’s already in place or by surrounding users. This obstacle disappears in a WLAN. Instead, user credentials and data are broadcast from both the client and the wireless access point (WAP) in a radius up to 500 feet. Broadcasting data via radiowaves rather than cables introduces the following security challenges:
  • How to prevent user credentials from being hijacked during authentication negotiation
  • How to protect the privacy of the data being transmitted between client and WAP
  • How to make sure the authorized user connects to the right network

     Security issues were addressed at Wayne State Medical School by making the exams accessible via a server. Each student PDA was registered and required authentication to access the exam. Security checks were conducted to ensure that no one could hack in from outside the wireless environment. Client and server software was developed for this project that supports a wide variety of 802.1x security methods. The client ran on Windows desktops, laptops and PDAs, and supported all wireless adapter cards that implement the standard set of NDIS 802.11 WLAN object identifiers. The server software was based on the RADIUS protocol. This server managed connections from WLAN clients to ensure that only authorized users can connect, and provide security information to the WAP so it can set up a private connection over the wireless link. The server ran on Windows XP/2000 and works with any 802.l1x WAP.

     Students entering the wireless testing environment required authentication to access the exam materials. Once the student logged-on to access the exam, their PDA was locked out of other applications that are stored on the device or might be retrieved from the web. As an additional layer of security, we used the recently developed proctoring console software. A host-client relationship between devices was established that allowed the exam administrators to monitor and control access to information on individual PDAs in the wireless environment. Using this application, we established an electronic monitoring system whereby exam proctors may view and record activity of the individual PDAs from a laptop computer located in the front of the lecture hall (Figure 1). If a test-taker was attempting to switch applications by accessing information stored on the PDA, an e-mail communication from the outside, or the internet, then the proctor was alerted to terminate access to the server for that particular device. In addition, the PDA activity of each test-taker was digitally recorded for later reference in the event of a dispute. The proctoring console software also provided a power meter for each device being monitored alerting exam administrators that a supplemental battery pack may be needed by some test-takers during the exam. Since the student PDAs were locked out of access to other applications during the exam, it was not possible for the proctors to view personal content stored on the device. Exam results were sent via e-mail following the session providing students and faculty with immediate performance feedback.

     Survey results revealed that 53% of the pilot group felt that preparation for the computer-based US Medical Licensure Exams was the biggest advantage of using the PDAs for assessment. Immediate availability of exam scores was the second favorite with 37% of the PDA group selecting this survey option. The other survey results pertaining to the use of wireless PDAs for exam administration are shown in Table 2. While the majority of the project participants did not find the proctoring console (70%) or browser lock-down (90%) functions serious distractions, most still preferred to take an exam using paper and pencil (70%) or a desktop computer (55%). Upon class-wide implementation of the wireless PDAs, we will compare the performance of students participating in on-line vs. paper and pencil exams.

Conclusions

      We have chosen to promote the use of PDAs by our undergraduates rather than laptop computers due to the potential for mobile classroom and clinical applications.12 Our pilot project has permitted us to test a variety of wireless implementations in the undergraduate medical curriculum prior to a large scale implementation. We gained insight into a variety of logistical issues, technical concerns, and student user styles. Approximately 25% of the students in our group embraced the technology and helped their classmates, relieving us of some support requirements. Another 25% of the PDA group was less technically adept and required personal support from the CampusMobility® team. On-site and remote technical support will be a necessity for class-wide implementation of the initiative.

     Plans for class-wide implementation of wireless PDAs at Wayne State Medical School have incorporated several lessons learned from the pilot project described in this report. A web-based survey revealed that 75% of our pilot group would be willing to spend up to $400 for the personal purchase of a PDA if the devices were not provided by the School of Medicine. This is a feasible price range for a large-scale purchase of PDAs. A list of supported devices and available software will be provided to the students to select from for personal purchase. There are plans to expand the wireless infrastructure throughout the medical school and library. Site licenses are being pursued for general- and discipline-specific medical software and faculty development support will be provided to adopt the interactive teaching methods introduced into the classroom by wireless technology.

     Mobile device management provided by CampusMobility® will be an essential component of the large-scale implementation that will be used to identify those devices that are not functioning properly or have outdated software prior to the wireless exams. In addition to this remote support, it will be necessary to provide on-site technical assistance to troubleshoot routine problems such as lost wireless access settings. It will also be necessary to optimize the graphical user interface design that fits the PDA screen and keeps navigation to a minimum. Convenient interfaces such as a table maker and expandable images are other successful components used during the pilot that will be included in the large-scale implementation. Because the Department of BioMedical Communications at Wayne State Medical School has developed web-based material for all basic science courses, it will be possible for all first and second year students to access relevant content, in addition to evaluations and schedules, using the wireless PDAs.

     Our survey results indicated that the most favorable aspect of the wireless PDAs was accessing web-based course content and medical software. These features were favored by 68% of the pilot group with e-mail communication favored by 16% of the participants. Access to medical software will be the feature most heavily utilized by undergraduate students in clinical rotations, residents, and practicing physicians.4-6 A majority of the pilot group (53%) felt that the most useful feature of the wireless handhelds for exam administration was preparation for the computer-based USMLE with immediate availability of performance favored by 37% of the participants. Although immediate test scores could be accomplished using computer-based exams, Wayne State Medical School currently lacks the facilities for desktop computers dedicated to testing. Therefore, the PDA-based exams have the potential to fulfill this need. Our survey revealed that the students still preferred paper and pencil exams, with 70% of the students strongly agreeing or agreeing with this option on our survey (Table 2). A post-exam interview indicated that this preference was a reflection of the students’ desire to flag questions during the exam for later review. This is a feature that will be built into the PDA-based exams in the future. Security issues involving testing in a wireless environment have been rigorously tested during this project. A redundant process involving authentication at log-in, browser lock-down, and the proctoring console made our wireless exams more secure that the traditional paper and pencil type.

     Our goal is to utilize the wireless mobile technology to enhance the educational experience of our undergraduates and to facilitate some administrative functions in at the School of Medicine. In the process, we are training future physicians to use a tool that is becoming increasingly popular in the medical community for accessing patient tracking applications, drug databases, and diagnostic algorithms.6,13

Acknowledgements

     This work was supported by a 2001-2002 Educational Development Award from the Office of the Provost, Wayne State University. The authors would like to thank Brad Wharry for his technical support and Laura Thomas for her inspiration.

References

  1. Mamary EM, Charles P. On-site to on-line: barriers to the use of computers for continuing education. J Continuing Education Health Professional 2000;20:171-5
  2. Manning PR, DeBakey L. Continuing medical education: the paradigm is changing. J Continuing Education Health Prof. 2001;1:46-54
  3. Ward JP, Gordon J, Field MJ, Lehmann HP. Communication and information technology in medical education. Lancet 2001;357:792-6
  4. Criswell, DF, Parchman ML. Handheld Computer Use in U.S. Family Practice Residency Programs. Journal of the American Medical Informatics Association 2002;9:80-86 http://www.jamia.org.
  5. Sullivan L, Halbach JL, Shu T. Using Personal Digital Assistants in a Family Medicine Clerkship. Academic Medicine 2001;76: 534-5 http://www.academicmedicine.org/content/vol76/issue5/index.shtml
  6. Stoddard M. Professional and Patient Use of Palm Tops, PDAs, and Handhelds. Arizona Health Sciences Library Health Care Journal Articles: The PDA Bibliography. October 2002 http://educ.ahsl.arizona.edu/pda/art.htm
  7. Bridge, PD, Ginsburg, KA.. An Integrated Approach for Evaluating Students’ Achievement of Clinical Objectives. Medical Education Online [serial online] 2001;6:9 http://www.med-ed-online.org
  8. Zimitat, C. Designing effective on-line continuing medical education. Med Teacher 2001;23:117-22
  9. USMLE Administration, Minimum Passing Scores, and Performance. 2000 USMLE Performance Data, National Board of Medical Examiners Annual Report http://www.usmle.org/news/medlic.htm
  10. Ioannidis J, Rubin AD. Breaking 802.11 Security. AT&T Research News 2001 http://www.research.att.com/news/2001/October/80211Security.html
  11. Stubblefield A, Ioannidis J, Rubin AD. Using the Fluhrer, Mantin, and Shamir Attack to Break WEP. AT&T Labs Technical Report; 2001: 1-12 http://www.cs.rice.edu/~astubble/wep/wep_attack.pdf
  12. Alur P, Fun CH, Rhoon KD, Eng KH, Joseph R, Hee LS, Hin LE. The Role of Wireless Pocket PCs in Medical Curriculum Delivery & Formative Assessment for Medical Students. Centre for Development of Teaching and Learning. CDT Link 2002;6 (1) http://www.cdtl.nus.edu.sg/link/mar2002/tech1.htm
  13. Feisal AA, Bedard PL. Palm reading: Handheld hardware and operating systems. Canadian Medical Association CMAJ 2002;167:775-80. http://www.cmaj.ca/cgi/content/full/167/7/775

Reference

Ganger AC, Jackson M. Wireless handheld computers in the preclinical undergraduate curriculum. Med Educ Online [serial online] 2003;8:3. Available from http://www.med-ed-online.org

Correspondence

Matt Jackson, Ph.D.
Associate Professor
Dept Immunology and Microbiology
Wayne State University School of Medicine
540 E. Canfield Ave.
Detroit MI 48201

(313) 577-1299
(313) 577-1155 FAX
mpjacks@med.wayne.edu

 

Medical Education Online Editor@Med-Ed-Online.org