Generating Productive Learning Issues in PBL Tutorials: An Exercise to Help Tutors Help Students*Kathleen M. Quinlan, Ph.D., Office of Educational Development, College of Veterinary Medicine, Cornell University
Problem-based learning (PBL) is structured to help students: 1) learn how to direct and manage their own learning; 2) develop their problem solving skills, particularly the clinical reasoning process, and 3) learn important principles and key concepts.1,2,3 In problem-based learning, students start with a discussion of a case or problem to generate "learning issues," which they research during substantial amounts of autonomous study time. Students prioritize what they need to learn, make choices about the resources they will consult, work collaboratively with colleagues, and organize their efforts to address learning issues in sufficient depth. Students must quickly learn how to manage their own learning in order to make good use of their tutorials and independent study time. Previous studies have shown that student-generated learning issues serve as the main starting point for students’ individual study,4 and help structure and direct the discussion in the reporting phase of the next tutorial 5 Thus, the development and refinement of learning issues during the tutorial is an important lever in helping students plan, organize and critically reflect on their independent study.6 Concerns are often raised about the degree to which problem-based learning is effective in achieving sufficient content coverage.7 Albanese and Mitchell7 suggest several strategies for ensuring content coverage, including sharing group learning issues and providing students with faculty-generated learning issues. These strategies, however, seem to sacrifice self-directed learning in order to achieve content objectives. Shahabudin, on the other hand, found congruence between student-generated learning issues and faculty-generated objectives as listed in tutor guides.8 Shahabudin attributed the finding to the guiding role of the tutor in the small group discussion, and students’ own abilities to identify their knowledge deficiencies. Consistent with Shahabudin’s conclusions, the exercise described in this paper was designed to focus tutors’ and students’ attention on developing productive learning issues in order to enhance students’ self-directed learning and consequently, their ability to thoroughly address important content objectives. Thus, it was meant to advance two of the key goals of problem-based learning, rather than compromising one for another. This paper describes the use of this exercise, "Developing Manageable Learning Issues," with tutors and students in a first year, problem-based learning course in veterinary medicine at Cornell University. The exercise (Appendix 1) was adapted from materials developed at the University of New Mexico Health Sciences Center School of Medicine for the Options in Health Sciences Education Workshops.9 The use of the exercise in a 45 minute workshop for tutors is described in enough detail to allow the reader to conduct such workshops. The tutor workshop was intended to promote reflection on the criteria for and importance of good learning issues, and to help tutors carry out a 15-20 minute mini-workshop with the students in their tutorial during a special extended tutorial in the middle of the first week of the course. The mini-workshop format within the tutorials arose for several reasons: 1) student (and tutor) attendance at separate educational process-oriented workshops for students has been poor in the College; 2) it allowed the criteria to be immediately applied to their group’s current work; and 3) it made the learning activity a shared group experience, making it more easily referred to, built on and supported by students and tutors throughout the course. Tutor feedback about the use of the exercise, perceived student reaction to the mini-workshop and the immediate effect of the mini-workshop was solicited during a meeting of the tutors immediately following the mini-workshop. Tutors who did not provide feedback during the group meeting were sent a follow up e-mail, asking them to comment on those areas. Tutors’ feedback is reported and analyzed in order to inform future use of this exercise with first year students. Participation rates also are analyzed in comparison to previous stand-alone workshops as a measure of the success of the activity. Use of the Exercise with Tutors The exercise was conducted as part of a refresher workshop for tutors who facilitate groups of six or seven students in Foundation Course III: Function and Dysfunction (for more information about Cornell’s curriculum or the content of this course, please see http://www.vet.cornell.edu/about/curric.htm10 or Quinlan11). Tutors then adapted the exercise for use with students in a mini-workshop during the first week of the course. The exercise for students was designed to achieve the following objectives:
The workshop for tutors (and the suggested formats for use with students) was consistent with Prosser and Trigwell’s12 proposal for a research-based approach to designing effective faculty development programs. They explicate two guiding tenets: 1) faculty need an opportunity to become aware of and discuss differing conceptions of teaching/learning and 2) the discussion needs to be seen as relevant to the faculty members’ own work. For the exercise, a sample case was chosen from the course and an actual set of students’ learning issues from the previous year was used. Tutors first ranked the learning issues from "best" to "worst" in terms of "how likely they were to lead to a productive discussion in the next tutorial session." No other guidance was given about how to rank the learning issues, since the point of the exercise was to prompt faculty to articulate and share their explicit or implicit beliefs about what makes a good learning issue. Individual rankings were then compared among the tutors for each of the learning issues. This comparison revealed substantial differences among individuals. Participants were asked to explain their rankings, focusing on contrasting rankings for a given item. From these justifications, criteria for good learning issues emerged. The criteria that the nine faculty participants developed are summarized below. The list should not be viewed as definitive and has not been empirically validated. Rather, it captures the collective values and experiences of the tutors who participated in the workshop. Workshop participants thought that good or productive learning issues are:
There was universal and immediate support among the participants for the idea that learning issues needed to be related to the case and the course (#1 and #2), should be specific (#3), and clearly stated (#4). However, there was more controversy and discussion generated in the process of developing the remaining criteria (#5-#8). The participants expressed enthusiasm for the opportunity to explicitly address what makes a good learning issue, noting that the exercise made them aware of a wider range of important criteria for productive learning issues, deepening their understanding of and attention to this critical aspect of the tutorial process. The sense of ownership of the criteria, engendered by the participatory nature of the exercise, also appeared to contribute to the tutors’ enthusiasm for and understanding of the criteria. The workshop took approximately 45 minutes with a group of nine tutors and an experienced facilitator. Four tutors were absent from the workshop and were briefed individually before conducting the exercise with their students. Tutor Feedback on Use of the Exercise with Students Tutors reported on their experience at a group debriefing shortly after facilitating their mini-workshops with their students. Six of the thirteen tutors volunteered comments during the debriefing. Those who did not participate in the discussion received individual, follow up correspondence, asking for their feedback. Four tutors responded to those individual requests. Three of the thirteen tutors did not respond in either the group or individually. With an understanding of the objectives, some tutors adapted the exercise to fit their own style. One tutor reported that he had given the students the above criteria, asked students to apply them in a critique of their own learning issues list and then rewrite their learning issues to better address the criteria. Other tutors asked students to rank, justify and develop criteria from their own learning issues list. Once those criteria were explicit, the students refined their learning issues to better address the explicit criteria. Two tutors indicated that the students went on to develop some of the "best learning issues" the tutors "had ever seen". One or two tutors commented that their students already seemed to be quite good at developing learning issues, implying that the exercise may not have been essential. Another tutor noted that the students seem to struggle to come up with good learning issues, but indicated that they were "ambivalent about conducting the exercise." Five tutors reported using the case and learning issues almost exactly as it was developed. To simplify the task and save time, one of those tutors asked students to simply pick the two best and the two worst learning issues, rather than rank all of them. Student-generated criteria were then applied to their own learning issues for the case of the week, with positive results. One explained, "My students enjoyed the learning issue exercise and we all learned something from it . . . . I learned that I prioritize learning issues from a very different perspective than the students do and they realized that it was something useful to learn how to do and spend more time on it. The students were willing to put the time and energy into that exercise because (I think) they could see the tangible benefits." In sum, the intended outcomes were achieved in many of the groups according to the reports from tutors . In addition, two of the tutors described positive, unexpected outcomes. In their discussions, major differences between students’ approaches to studying emerged as a result of the exercise. The exercise afforded an opportunity to talk about individual differences in approaches to the case, which heightened students’ and tutors’ appreciation for the different strengths and perspectives students brought to the tutorial. Discussion From tutor feedback, the exercise appeared to be generally successful. Reflection on tutor comments revealed several areas for improvement. First, tutors struggled with the example used in the exercise because some tutors were familiar with the teaching case and ranked the learning issues based on their knowledge of the full case. Other tutors were not familiar with the full case and rated the learning issues solely on the basis of what was provided in the synopsis. These led to spurious differences in rankings that led to some confusion and criticism from the tutors. While criteria were successfully generated through the exercise, these differences in prior knowledge in relation to the case created a distraction during the workshop. In the future, it may be desirable to eliminate that distraction or clarify the purpose up front so that participants will focus more on the criteria and justifications than the actual rankings. Second, the time allotted for the mini- workshop with students may be too short. Time factors were mentioned by a couple of tutors. Asking students to pick out only the two best and the two worst learning issues may be a way to save time, rather than ranking all of the learning issues. Allowing 30 to 45 minutes for the mini-workshop may also be warranted to allow students to complete all aspects of the exercise, including applying their criteria to the learning issues they generated that day or week. Several tutors also suggested that the exercise may be more useful to students earlier in their first year. The fact that some tutors felt that students were already adequately skilled and aware of the significance of their learning issues in shaping their learning experience also indicates that the impact of the exercise might be stronger earlier in the curriculum. Although noncompliance and variable tutor skill at facilitation are inherent disadvantages of the mini-workshop format integrated within the tutorials, the format seemed to be more effective at reaching the majority of students. Even if all three of the tutors who did not volunteer feedback did not comply, this format still reached more than three-quarters of the class. That proportion is substantially larger than the ten percent attendance achieved with the same cohort of students at two previous stand-alone, add-on process workshops on different, but related topics. The study was limited in that it relied only on tutor reports. Further research could test the effectiveness of the exercise in improving students’ learning issues, relying on a direct analysis of the quality of the learning issues using a pre- and post-test, randomized control group design. However, such a study would be complicated by issues of treatment fidelity, the small number of groups involved (since the group would be the unit of analysis, not the individuals) and ethical concerns about providing all students equal access to a potentially beneficial tutorial exercise. Alternatively, students (or tutors) could be asked to individually generate a list of criteria (or a concept map) for productive learning issues before and after the mini-workshop to determine whether they were able to articulate a wider range (or more refined set) of criteria as a result of the workshop. This approach also allows for closer inspection of the way that students think about learning issues and, therefore, could simultaneously enhance the educational value of the exercise while gathering research data. Finally, students’ ratings of the value of the exercise could be obtained through a brief survey. This paper offers a preliminary description of and reflection on an exercise conducted with tutors and students to generate more productive learning issues. This activity illustrates one way to "scaffold" students’ ability to manage their own learning, a vital aspect of PBL that Greening argues is often overlooked.13 Further research could focus on other methods of building the skills that students need to effectively learn in a problem-based curriculum. Such descriptive reports, even at preliminary stages, may be useful to tutors and others who work with students in a problem-based learning curriculum. References
Appendix: Developing Manageable Learning Issues Exercise Quinlan KM. Generating Productive Learning Issues in PBL Tutorials: An Exercise to Help Tutors Help Students. Med Educ Online [serial online] 2000;5:4. Available from URL http://www.med-ed-online.org. Send all correspondence to: Kathleen M. Quinlan, Ph.D. Phone: (607) 253-3767 |