Evaluation of World Wide Web-based Lessons for a First Year Dental Biochemistry Course

Alan E. Levine, Ph.D.

Department of Basic Sciences
The University of Texas Dental Branch at Houston
6516 M. D. Anderson Boulevard
Houston, TX 77030

Abstract: First year dental students at The University of Texas Dental Branch at Houston (Dental Branch) are required to take a basic biochemistry course. To facilitate learning and allow student self-assessment of their progress, WWW-based lessons covering intermediary metabolism were developed as a supplement to traditional lectures. Lesson design combined text, graphics, and animations and included learner control, links to other learning resources, and practice exercises and exams with immediate feedback. Results from an on-line questionnaire completed by students in two different classes showed that they completed 50% of the lessons and spent an average of 4 hrs. on-line. A majority of the students either agreed or strongly agreed that practice exercises were helpful, that the ability to control the pace of the lessons was important, that the lesson structure and presentation was easy to follow, that the illustrations, animations, and hyperlinks were helpful, and that the lessons were effective as a review. The very positive response to the WWW-based lessons indicates the usefulness of this approach as a study aid for dental students.

Key words: biochemistry, metabolism, dental education, computer assisted learning, World Wide Web, teaching aid, Internet resource.


     A challenge for today’s educators, including those involved in dental education, is to help the student learn rather than just teach. Development of critical thinking skills and the ability to be a life-long learner are required for a successful dental career. The development of computer technology and its incorporation into the learning process allow these goals to be more easily met. Developers of online courseware have taken a variety of approaches to facilitate learning and make it more accessible to the student. Textbooks are now often accompanied by CD-ROMs and/or links to a publishers website that contain additional material including enhanced versions of material that could not be represented in a printed textbook. This includes animations and 3-dimensional representations of anatomic features, cells, and proteins for example. Complete textbooks can be entirely published online.1,2 Stand alone CD-ROMs also have been published on a variety of subjects and many instructors post material such as lecture notes, answers to problems, and complete lessons on the WWW. All of these media can be used as supplements to the traditional lecture/laboratory course or can be used to deliver the course without any traditional in-class lectures. The key to all of these varied types of computer assisted learning is interaction; either with the content or between the student and other students or instructors.3 This technology can promote “learning to learn in situ”.4 In almost all cases where an assessment was conducted, the computer-assisted education was at least as effective in instructing students as traditional lectures.5-7 However, many factors can contribute to measured outcomes and a pragmatic approach in evaluation of educational outcomes focuses on students’ perception of their educational experiences.8

A wide variety of didactic, clinical, and continuing education courses in dental education are currently available on the WWW. Many of these have been carefully evaluated and shown to be effective teaching/learning tools for dental education.3 These include courses in dental anatomy9, endodontics,10 orthodontics11, prosthodontics12, and radiography.13 The computer program in the dental anatomy course focused on tooth recognition, and students using this program scored better in both didactic and practical examinations.9 An online course in dental terminology for dental hygiene students, containing laboratory exercises in addition to content pages, was shown to be as effective as traditional classroom instruction.14 Several examples of biochemistry courses online also exist. Rourk15 describes the use of a CD-ROM together with a textbook to better represent aspects of biochemistry as 3D models. The Books4doctors site4 lists four biochemistry, four molecular biology, and several other biochemistry-related texts as available for free online through their website. Recently, large numbers of continuing education (CE) courses have become available on-line.16,17 An extensive evaluation of nine different online CE courses indicated that most users gained new knowledge and obtained an increased understanding of the material.18,19

     A model developed for the implementation of online courseware, the hypertutor or hypermedia model, encourages the learning process by incorporating five specific instructional parameters.20-22 These are learner control, presentation, practice, feedback, and added learning resources. These hypermedia programs allow both linear and non-linear navigation through the material and have been shown to benefit the students regardless of their learning styles.3,20,23,26 The inclusion of hyperlinks to outside sources makes access to a broad range of learning resources possible and lets the user access these resources as needed almost instantaneously. Although the more experienced user is more likely to choose a non-linear pathway25 all users are thus empowered to make their own decisions as to what resources they need to master the subject matter.23-25 Elaborative learning and reinforcement are both important characteristics of most computer-based programs. These techniques have been shown to be useful methods for increasing student’s retention of learned materials.26

      A course in biochemistry is required for all students by the Dental Branch to help meet the American Dental Association accreditation standards in biomedical sciences instruction. This course is a difficult subject for first year dental students and as taught at the Dental Branch is very time intensive. There is not enough in-class time or study time for most students to master the material even though biochemistry is a prerequisite for admission to the Dental Branch. Therefore, new methodologies to enhance the student’s ability to master the subject were investigated and the decision made to use WWW-based lessons as a supplement to the in-class lectures. A series of lessons covering the metabolism section of Dental Biochemistry 1521 were developed following the hypertutorial model to answer the following research question. “How effective are online lessons as a supplement to traditional lectures?”

Methods

     Development of WWW-based Lessons - Development of the WWW-based biochemistry lessons was done at the Dental Branch. The majority of the lessons, which were developed by the author, were written in Hypertext Markup Language (HTML) using a basic text editor. Form elements and interactive exercises and examinations were programmed using JavaScript. The lessons comprise approximately 275 HTML and image files for a total of 3 megabytes. A web-based lesson was defined as an interactive way of presenting material to the student. The lesson combines text, graphics, and other multimedia techniques to improve the learning experience. An important feature is learner control via imbedded questions with immediate feedback. The material from the lectures is broken into smaller “packets” to improve the ease of studying the material. Key words within the lessons were hyperlinked to a glossary developed as a separate HTML document, specifically for these lessons. The Table of Contents for the lessons is shown in Figure 1 contained in the appendix. For the pilot study, only lessons 1-8 were available to the students. This was followed by the 2 years of the main study where all lessons were available and reviewed by the students.

     Instructions to Students - Students enrolled in the Biochemistry course were given the URL for the lessons and instructions regarding completing lessons and the evaluation in order to gain credit for their work. Students were awarded 5 points (total course comprises 700 points) for completing lessons and the evaluation. Because of problems encountered with emailing the evaluation form to the instructor, some students turned in a printed copy of the evaluation. The student’s name was recorded to give them credit and then removed from the evaluation form so that the results of the evaluation were tabulated anonymously. The lessons are now password protected.

     Assessment Tool - The on-line evaluation form used to obtain student responses contained both Likert scale questions and free text questions. The anchor statements for the Likert scale were: Strongly agree, Agree, No difference, Disagree, Strongly disagree. This tool contained questions about lesson structure (3 questions) and content (5 questions), the value of practice exercises (3 questions), their general impressions of the web-based instruction (7 questions), and asked the students to estimate the time spent and percentage of lessons they completed. One question asked the students for their “level of experience” with computers. In addition, the students were asked for “what they liked best,” “what they liked least,” and any additional comments that could be answered as free text. Student’s responses to surveys of time spent online have been shown to be generally substantiated by server statistics.27 The assessment tool was reviewed by several UT-Houston faculty experts in this type of evaluation process to assess its face validity. In addition, a pilot study was conducted with a sample of students (6 of 63 students in the 1998/1999 class) to further refine the assessment tool and the lessons.

     Responses to the Likert scale questions were tabulated and reported as % of respondents giving a certain response. Student responses to the free-text questions were analyzed for major themes and the frequency of response for each theme recorded.

     Sample Size - For the full evaluation, the 1999/2000 class, 52 of 61 students (85%) enrolled completed some lessons and the evaluation while 98% of the students (61/62) from the 2000/2001 class completed some lessons and the evaluation. The process of quantifying the number of lessons completed is described fully in the Results section.

Results

     Students enrolled in Dental Biochemistry 1521 were given access to all of the lessons and instructions in their use at the beginning of the lectures on intermediary metabolism (the subject of the WWW-lessons). The students were told that they could receive credit by completing the questionnaire even if they did not find the WWW-lessons useful.

     The students spent approximately four hours using the WWW-based lessons and completed approximately half of the material (Table 1). The wide variation in the use and completion rate of the lessons probably reflected several factors including the fact that the lessons were not mandatory, no fixed amount of time was required to obtain credit, and the fact that all required learning objectives were contained in the traditional lectures and handouts. In addition, students’ background and knowledge of biochemistry varied widely depending mainly on the undergraduate institution attended and this probably influenced the extent of time spent on the lessons.

 

      The student responses to the questionnaire are tabulated in Table 2. The results obtained from the two classes were very similar. Greater than 90% of the students agreed or strongly agreed that the directions for using the lessons were clear, that the practice exercises were helpful, that the ability to control pace of the lessons was important, and that more Basic Science courses should be available on the WWW. Approximately 60-80% agreed or strongly agreed that the structure and presentation of the lessons was easy to follow, that the lessons were interesting and were effective as a review. A similar number agreed or strongly agreed that the illustrations and animations were helpful and that the hyperlinks to the internal glossary and outside sources were helpful. About 50% agreed or strongly agreed that the lessons emphasized what the student needed to know and that the amount of material presented in each lesson was appropriate. In almost all of cases the number of students replying “no response” was <3%.

     Approximately 45% of the students agreed or strongly agreed that the lessons would be useful for study for the Dental National Board, Part I Exam, while 23% indicated it would not make a difference and 30% had “no response.” At the Dental Branch, students take the National Board, Part I Exam after their second year. Therefore, the relatively high level of “no response” reflects the student’s unfamiliarity with the National Board Exams during their first semester in dental school. Further study of the students during their review for these exams would be necessary to adequately assess the usefulness of the WWW-lessons in the review process.

     As indicated by Table 3, greater than 90% of the students in both classes indicated that they used computers at least occasionally. In the 1999/2000 class, 58% of the students indicated that they were very familiar with computers, using them for a variety of tasks. This number was higher (79%) in the 2000/2001 class.

      The student comments to the free-text question “What was liked best” on the evaluation form for the 1999/2000 and 2000/2001 classes were similar and, in general, elaborated on the answers given in the scaled section of the form. However, two themes stand out because of the frequency with which they were mentioned by both classes. The first is the “ability of the student to set their own pace,” which was included in 33% of the forms from each class, and the second was the “helpfulness of the embedded questions” (included on approximately 25% of the forms). The availability of immediate feedback for the questions was also cited as being very helpful in studying (approximately 15% of the students). Comments regarding the use of color, animations, and diagrams were listed by 5-10% of the students in the two years. Some comments were made by only one or two students, but were often very useful for evaluation and future development. Several of these themes listed by single students reflected learning style, for example, and that they appreciated the material being presented in a different way. One student specifically stated that he/she was a visual learner. Although most students liked the web-based lessons, several (5%) commented that they did not want web-based lessons to replace the traditional lectures. Approximately 10% felt that more of the Biochemistry course as well as the other Basic Science courses in the curriculum should have material available on the Internet.

     Students were asked to provide feedback regarding “what they liked least” and how the instructional modules could be improved. The majority of these comments given in the free text section of the evaluation form were related to problems with Internet access from home, the configuration of the computers in the Dental Branch Learning Resource Center, and with sending email responses to the instructor (20% in the 1999/2000 class and 25% in the 2000/2001 class). Approximately 33% of each class commented that the imbedded questions were too easy and that these questions did not reflect the difficulty of questions given on the in-class examinations. Five percent of the students felt that it was difficult to determine which sections of the lessons were the most important.

Discussion

     A series of WWW-based Biochemistry lessons covering intermediary metabolism were developed for use by first year dental students at the Dental Branch. The results obtained from two different first year Dental Branch classes were very positive. In agreement with many other studies,4,12,24,28-30 the most important advantage of the use of these lessons, as indicated in the student evaluations, was the ability to control the pace with which the lessons were used. The interactivity of the lessons and the review questions and exams with immediate feedback was also repeatedly cited by the students as important features. The availability of “sample test questions” was important and the one consistent comment from the students was that they wanted more questions that were representative of in-class exam questions.

     One factor that could influence the student’s perception of the usefulness of the lessons is their familiarity with the use of computers. All students complete a course entitled “Information Resources Training” in the week prior to the start of formal classes at the Dental Branch. Greater than 60% reported using computers for a variety of tasks. Therefore, familiarity with the use of computers probably did not influence the students’ use of the WWW-based lessons.

     Most of the student comments in the free-text section of the evaluation elaborated on the responses to the objective section. However, several themes were very insightful in understanding the value of such WWW-based lessons to Dental School education. For example, statements were made related to different learning styles and were consistent with the many theories that students learn in different ways. One student directly stated that he/she was a “visual learner” and was helped by being able to read over the material as many times as needed.

     Feedback contained in the free-text comments from the students led to improvements in the WWW-based lessons. In response to these comments from the 1999/2000 class, “practice exams” modeled after questions from previous years’ exams were included for the 2000/2001 class. These questions gave immediate feedback to the students regarding the correct answer and explanations were provided for many correct and incorrect answers as appropriate. Even with this added resource, the students in the 2000/2001 class wanted more practice exercises. Student responses make it clear that the more of these sample exam questions that are available, the better. Several comments also made it clear that the web-based lessons were perceived as a very useful resource for studying for this course and for the National Board exams in the future.

     In dental education, little literature exists concerning the incorporation of critical thinking and life-long learning and even less regarding the use of information technology in dental education to promote these ideas. Johnson and Schleyer have argued for the need for a set of guidelines to guide developers in the production of high quality instructional material that meets the students needs.31

      It is important to understand how the use of online education influences communication skills and interactions between the students and between students and instructors.32 When the evaluation focus is on students’ perception of the process, the outcomes are generally very positive in favor of the incorporation of technology into the learning process.8 How technology is used can make the difference between meaningful and superficial learning.33 Although such studies are in their infancy, it is clear that there is an ongoing shift from a teaching to a learning paradigm.3,24,34,35

     Problem-based learning, case-based learning, and online learning all require a higher degree of student involvement in the learning process. Active learning must be the primary goal for the students to benefit from WWW-based materials36 as increased active learning has been shown to correlate with increased understanding .22 This shift from a passive to an active learning process necessitates a change in the pedagogical practices of instructors. Preliminary data for a Biology course37 and a “structure of the Human Body” 27 course indicated that there was a correlation between student grades and the number of times students accessed course material via the WWW. The correlation was even better when only students receiving final grades in the upper 33% of the class were considered.27

      Based on the results of the present study, dental students actively embrace the online learning paradigm and want more of this type of learning in the dental curriculum. However, instructors must ensure that the goal of enhancing and improving the educational process not be overlooked just to bring technology into the curriculum.8,35 Computer-based learning must be more than an extension of the overhead projector or blackboard. These materials must do more than reduce the old curriculum to a new media24,28 The evolving literacy of being able to do “information navigation” rather than only text/figure navigation as found in traditional lectures and textbooks must be incorporated.4 Allowing the learner to learn in situ is an important aspect of the new teaching pedagogy.4

     Computer-based instruction can serve as a catalyst for change in many areas of teaching/learning not directly associated with technology. Yazon et al28 noted an increased interactivity by students in class that they attributed to utilization of on-line resources including asynchronous chat rooms. A change in thinking about subject matter and its presentation on-line can translate into better ways of presenting material in class. This can include reorganization of lecture topics, use of better means of presenting factual information (tables vs. linear statements), and the creation and inclusion of new graphics or animations in lectures to engage the students. In addition, the opportunity to integrate topics also lends itself to web-based lessons29 since a common topic can be linked to a variety of related topics. Several students in the present study commented that they did not want to have WWW-based lessons replace in-class lectures. A balance between face-to-face teaching and technology based teaching and learning must be maintained.3,21,30

      Continuous updating and improvement are required to keep the WWW-based lessons current and the positive student responses indicate that this is time well-spent by the instructor/web-author. An important goal for the future of these WWW-lessons will be to continue to add more representative questions to the practice exams. Integration of WWW-materials accompanied by the posting of digitized lecture materials should greatly enhance the utility of both resources for the student.

     The results presented in this study indicate that WWW-based lessons can serve as an important study aide to dental students taking a biochemistry course. Further research is necessary to determine other uses for the WWW-based biochemistry lessons. For example, students needing to remediate sections of a course could review relevant sections without having to go through the entire course again. In addition, these lessons are ideal for review for the National Board Part I Exams. The ability to navigate through the material in a non-linear fashion is ideal for review purposes. These lessons could also be used in the clinical setting either within the dental school or in private practice to provide the basic science background to a clinical problem. Further development of biochemistry and other basic science material could also be used in the distance learning and continuing education setting.

Acknowledgements

     This work was begun as part of a Fellowship sponsored by the Office of the Executive Vice President for Research and Academic Affairs (The University of Texas Health Science Center at Houston). The facilitators of the Fellowship, Drs. Allan J. Abedor and Craig W. Johnson (The University of Texas School of Health Information Sciences at Houston) were invaluable resources throughout this project.

References

  1. Bio 101 meets the internet. Science 2001;291:19.
  2. Freebooks4doctors! http://www.freebooks4doctors.com/fb/index.htm.
  3. Schittek M, Mattheos N, Lyon HC, Attström, R. Computer assisted learning. A Review. Eur J Dent Educ 2001;5:93–100.
  4. Brown JS. Growing up digital: How the web changes work, education, and the ways people learn. Changes 2000;March/April:11-20.
  5. Brown DG. The jury is in! Computer-enhanced instruction works. Syllabus 2000; August:22.
  6. Hallgren RC, Parkhurst PE, Monson CL, Crewe NM. An interactive, web-based tool for learning anatomical landmarks. Academic Med 2002;77:263-265.
  7. Brown K. Online, on campus: Proceed with caution. Science 2001;293:1617-1619.
  8. Lechner SK. Evaluation of teaching and learning strategies. Med Educ Online (http://www.med-ed-online.org) 2001;6:1-5.
  9. Wallen ES, Schulein TM, Johnson LA. A computer program to aid in visual concept development in dentistry. Comput Methods Programs Biomed 1997;52: 105-115.
  10. Plasschaert AJ, Cailleteau JG, Verdonschot EH. The effect of a multimedia interactive tutorial on learning endodontic problem solving. Eur J Dent Educ 1997;1:66-69.
  11. Shellhart WC, Oesterle LJ. Assessment of CD-ROM technology in classroom teaching. J Dent Educ 1997;61:817-820
  12. Pilcher ES. Students’ evaluation of online course materials in fixed prosthodontics: a case study. Eur j Dent Ed 2001;5:53-59.
  13. Ludlow JB, Platin E. A comparison of web page and slide/tape for instruction in periapical and panoramic radiographic anatomy. J Dent Educ 2000;64:269-275.
  14. Grimes EB. Effectiveness of an online course in dental terminology. J Dent Ed 2001;65:242-247.
  15. Rourk W. Virtual biochemistry - a case study. Future Generation Computer Systems 2000;17:7-14.
  16. Schleyer TK. Online continuing dental education. J Am Dent Assoc 1999;130:848-854.
  17. Dentalxchange: CE Online. http://www.dentalxchange.com/ce/ce.jsp
  18. Schleyer T, Johnson LA, Pham T. Instructional characteristics of online continuing education courses. Quint Int 1999;30:755-762.
  19. Spallek H, Pilcher E, Lee J-Y, Schleyer T. Evaluation of Web-based dental CE courses. J Dent Ed 2002;66:393-413.
  20. Johnson CW. Web-based Hypermedia Courseware Development with HTML and JavaScript: A Hands-On Introduction. 1997; http://www.uth.tmc.edu/uth_orgs/educ_dev/whcd/CH4/CH4.HTM#TOC
  21. Bates T. Teaching, learning, and the impact of multimedia technologies. Educause 2000;September/October:38-43.
  22. Stokstad E. Reintroducing the intro course. Science 2001;293:1608-1610.
  23. Kraus LA, Reed WM, Fitzgerald GE. The effects of learning style and hypermedia prior experience on behavioral disorders knowledge and time on task: a case-based hypermedia environment. Computers Human Behavior 2001;17:125-140.
  24. Lin B, Hsieh C-t. Web-based teaching and learner control: a research review. Computers & Education 2001;37:377-386.
  25. Reed WM, Oughton JM, Ayersman DJ, Ervin JR Jr, Giessler SF. Computer experience, learning style, and hypermedia navigation. Computers in Human Behavior 2000;16:609-628.
  26. Policher VE, Bagwell C. Pedagogical principles of learning in the online environment. Syllabus 2000;May:52-56.
  27. McNulty JA, Halama J, Dauzvardis MF, Espiritu B. Evaluation of web-based computer-aided instruction in a basic science course. Academic Med 2000;75:59-65
  28. Yazon JMO, Mayer-Smith JA, Redfield RJ. Does the medium change the message? The impact of a web-based genetics course on university students’ perspectives on learning and teaching. Computers & Education 2002;38:267–285.
  29. Parker MJ, Seifter JL. An interactive, web-based learning environment for pathophysiology Academic Med 2001;76:550.
  30. Reid WA, Harvey J, Watson GR, Luqmani R, Harkin PJ, Arends MJ. Medical student appraisal of interactive computer-assisted learning programs imbedded in a general pathology course. J. Pathol 2000;191:462-465.
  31. Johnson LA, Schleyer T. Development of standards for the design of educational software. Quint Int 1999;30:763-768.
  32. Farrington G, Bronack S. Sink or swim. T.H.E. Journal Online 2001;http://www.thejournal.com/magazine/vault/articleprintversion.cfm?aid=3484
  33. Garrison DR. A cognitive constructivist view of distance education: an analysis of teaching-learning assumptions. Distance Ed 1993; 14:199-211.
  34. Bork A. Learning technology. Educause 2000;January/February:74-81.
  35. Okamoto T, Cristea, A, Kayama, M. Future integrated learning environments with multimedia. J Comp Assist Learn 2001;17:4-12.
  36. Freberg L. Integrating internet resources into the higher education classroom. Syllabus 2000;March:48-50.
  37. Stith B. Web-enhanced lecture course scores big with students and faculty. T.H.E. Journal 2000;March:21-28.

Reference

Levine AE. Evaluation of World Wide Web-based lessons for a first year dental biochemistry course. Med Educ Online [serial online] 2002;7:13. Available from http://www.med-ed-online.org.

Correspondence

Dr. Alan E. Levine
Associate Professor
Department of Basic Sciences
The University of Texas Dental Branch at Houston
6516 M. D. Anderson Boulevard Suite 4.127E
Houston, TX 77030
Telephone: (713) 500-4497
FAX: (713) 500-4500
Email: Alan.E.Levine@uth.tmc.edu

 

Appendix

Figure 1

  

Medical Education Online Editor@Med-Ed-Online.org