Scaffolding for Success in Problem-Based Learning*

Tony Greening
School of Information Technology and Mathematical Sciences
The University of Ballarat, Australia

Abstract: Problem-Based Learning (PBL) is based on an alternative pedagogical model to the conventional, didactic one, and offers benefits to the quality of student learning. The approach has been adopted by many institutions. The focus of this paper is on the influence of the learning support structure in an environment (such as the typical PBL environment) that encourages student independence as one of its basic tenets. The immediate reaction might be to assume that it takes on a reduced importance in such circumstances. That assumption is challenged in this paper.

    The concept of "scaffolding" may take many forms¹; here it is used in a wide context to refer to all forms of learning support, whether or not they be cognitively-based, logistical, etc. Thus, the term is used to broadly refer to the range of services provided to assist learning. This is of interest as Problem-Based Learning (PBL) approaches to education suggest a strong role for factors such as authenticity and student independence. With the shift of emphasis that moving towards a PBL approach implies, a re-examination of the nature of what it means to offer meaningful support to learners is important. In this paper the relationship of scaffolding to the success of PBL programs is examined. In this context, "success" is measured largely in terms of those benefits with which PBL is associated, principally in the encouragement of "deep" (or personally meaningful and potentially transformative) approaches to learning. Elsewhere, deep approaches to learning have been described in terms of higher levels of integration of knowledge and greater tolerance to complexity².

    The paper begins by briefly referring to the definitional dilemma that hinders attempts to compare PBL programs. A working definition is established. Desirable outcomes of PBL are then presented, with an emphasis on "deep" learning approaches. This is then countered by raising some concerns about the performance of existing PBL programs, while hinting that appropriate scaffolding may offer relief from some of these concerns. An account of the role of the tutor in PBL follows, as an obvious source of scaffolding. Scaffolding is then discussed in terms of improving the likelihood of successfully encouraging meaningful approaches to learning in PBL students.

Defining PBL

    A review of the literature reveals an extensive coverage of PBL case studies, many based on differing concepts of PBL.¹ These concepts may directly affect the reported success or failure of the approach; for example, some authors refer to PBL as a mandate for "discovery learning" and then report that this introduces implementation problems in terms of required subject-matter coverage.³ Furthermore, as a reaction to this concern there is a risk of staff enforcing rigid course objectives to ensure subject-matter coverage, with a return to what the authors refer to as "curriculopathy" (disease of the curriculum).

    Although some variation in what is labeled as "PBL" may be aberrant, it is this author's observation that much of PBL's wide range of possible manifestations is to some extent inherent to the approach, as one which does not prescribe fixed and narrow pathways for learning. However, there are some essential criteria that constrain the definition of PBL. Importantly, PBL is identified as a constructivist pedagogy. Savery and Duffy⁴ summarize some of the central tenets of constructivism:

• Understanding is based on experiences with content, context, the learner's goals, etc., and these factors are inextricably woven together. Thus, understanding is a construction that is unique to the individual.
• Meaning is not transmitted, although it may be tested for compatibility with the meanings of others. From another perspective, cognition may be regarded as being distributed rather than individually localized.
• Puzzlement is the factor that motivates learning.
• Social negotiation and the ongoing testing of the viability of existing concepts in the face of personal experience are the principle forces involved in the evolution of knowledge.

    The definition of PBL that is used here is that it is a subset of problem-centered learning methods which is easily identifiable by the use of typically ill-structured problems which precede and motivate learning, and act as a vehicle for encouraging student ownership of the learning environment. Usually there is an emphasis on contextualization of the learning scenario, providing a basis for later transference, and the learning is accompanied by reflection as an important meta-cognitive aspect of PBL. Implementation of PBL programs usually occurs via group-based work, reflecting the constructivist focus on the value of negotiated meaning. A characteristic of PBL which follows from many of these attributes is that it is unconfined by discipline boundaries, encouraging an integrative approach to learning which is based on requirements of the problem as perceived by the learners themselves. Schmidt⁵ summarizes PBL in terms of three essential principles:

1. Activation of prior-learning via the problem;
2. Encoding specificity such that the resemblance of the problem to intended application domains facilitates later transfer (leading to an emphasis on authentic learning environments); and
3. Elaboration of knowledge via discussion and reflection to consolidate learning experiences.

Desirable Outcomes of PBL: Deep Learning Approaches

    Traditional approaches to preclinical medical education have been criticized as devoid of cross-disciplinary integration, having insufficient interface with clinical problems, and exhibiting insufficient retention of basic knowledge and students' inability to apply to actual cases, as well as concerns over accuracy of the knowledge base.⁶

    As an alternative to traditional methods, Robbs and Merideth⁷ list a number of advantages that are associated with PBL modes of learning:

• an increased retention of information;
• the development of an integrated (rather than discipline-bound) knowledge base;
• an encouragement towards lifelong learning;
• a greater exposure to clinical experience and at an earlier stage in the curriculum;
• an increased student-staff liaison; and
• an increase in overall motivation.

    Marton and Saljo⁸ and the Gotteborg educational research group⁹ have focussed on qualitative differences in individual approaches to learning, with the terms "surface" and "deep" used to reflect the degree of engagement and depth of understanding of the material. The word "approach" is emphasized here to indicate that this is not necessarily inherent to the learner, but may be variously (and possibly even strategically) applied by students in response to external factors. This establishes a position where the learning environment shares in the responsibility for encouraging more meaningful ("deep") approaches to learning.

    Problem-Based Learning (PBL) has been claimed to encourage deep learning in students.^10,11. These studies indicate increased use of meaningful ("deep") approaches by PBL students in relating to the material, and decreased use of reproductive ("shallow") approaches.

    PBL incorporates a number of factors that seek to encourage this meaning-orientation to learning approaches, such as high motivation, learner-centeredness, and group-based activity requiring peer negotiation.¹² A study by Sobral¹³ generally supports these propositions, finding that "the results ... reinforce the idea that problem-based learning, even in a single-course experience, may enhance the emotional well-being of the participants and the quality of the learning environment, as expressed by the learners themselves" (p.100). The current review relies heavily (although not exclusively) on student self-perception of the quality of learning, due to the availability of such information in the literature. It is reasonable to cast these student perceptions about quality of learning in the company of other perceptions in order to develop a feel for possible relationships that may be worth exploring. While this paper is primarily concerned with relationships between scaffolding and deep learning, it may not be possible to do so in complete isolation of other factors.

    There is evidence in the literature for relationships between quality of learning and other factors. For example, Kember and Gow¹⁴ summarize arguments which find correlations between reproductive learning approaches and such factors as high workload, assessment methods which reward recall, low levels of motivation in the course, and overly-constrained learning environments.

    PBL has a strong reputation for enhancing student motivation towards learning tasks and providing an unconstrained environment by means of student empowerment, thereby removing two of these possible indicators of shallow learning approaches. However, it is the author's opinion that PBL occasionally occurs in the presence of assessment schemes that are inappropriate to its orientation, and it is well documented that PBL may involve perceptions of increased workload. These areas will be explored here.

    Ma¹⁵ surveyed 3rd-year students of database-systems using Biggs' Study Process Questionnaire¹⁶, and compared the results with the outcomes of PBL projects. He found a linear relationship between good student outcomes in the projects and indicators of deep or achieving learning processes, and conversely, shallow learning strategies were found to be negatively correlated with good marks. However, this relationship is not explored, and the report suggests that more evidence is needed before claims can be made for correlations between deep learners and better performance in PBL projects.

    The increased motivation associated with PBL and evident in student surveys and staff observations of increased student attentiveness^17-20 is often regarded as an important outcome. Certainly, increased motivation is a desirable result. However, in this survey motivation is given a different emphasis. First, the focus is on quality of learning. Second, motivation is regarded here as essential for the success of PBL in the face of some of its strong demands, and therefore is treated more as input rather than an output. This will be discussed further in the context of the cognitive demands made by constructivist approaches to pedagogy.

Concerns Raised in Response to PBL Cases Studies

    The literature review conducted by Albanese and Mitchell¹ indicates a number of issues that may be of concern, as evidenced in the case-study literature. In the current review such issues are loosely classified in terms of suggested explanations for them: problems inherent to a process of change, problems emerging from traditional means of assessment, and problems relating to inherent difficulties with PBL as a mode of learning.

Issues Inherent to Change

    Pereira, and colleagues¹⁷ identify a common failing with PBL programs due to entrenched (non-constructivist) models of learning and power relations. The change to PBL represents a disruption to these existing assumptions, which then results in resistance:¹⁷ "Immediate and/or long term resistance is therefore a general and foreseeable reaction threatening sustainable reform" (p.352). The authors note an improvement in student evaluation of the course over time, however. O'Hanlon, Winefield, Hejka and Chur-Hansen²¹ refer to the transitional problems associated with students moving to a group-based learning environment, especially where prior academic performances were obtained via individual success on a competitive basis. Moving from a secondary school environment to a tertiary PBL first-year program is an example of this sort of anxiety-inducing transition. In the case of PBL, the degree of change is quite dramatic; despite the appropriateness of PBL as pedagogy in terms of current positions on human learning, Camp²² still refers the change to PBL as a "paradigm shift". This indicates differences between it and traditional approaches to education at very fundamental levels.

    Kenley²³ claims that professional domains such as medicine, architecture, etc., are inherently teacher-centered in that great emphasis is based on the knowledge accumulated in the teacher, which is often more highly regarded than the knowledge available in reference works. Kember and Gow¹⁴ suggest that lecturers that adopt highly instructivist approaches to teaching tend to see personal subject-based knowledge as the prime attribute of the professorate, and (therefore) regard their courses as a means of training of students for their professional roles. In terms of building and architecture, Kenley²³ describes the desired attributes of the teacher in such a system:

Thus the ideal candidate for teaching management of construction is someone who has learnt all the relevant material during their own education and undertaken continuing education during a long and successful career which has included a wide range of experience of all facets of the industry. For good measure they are an experienced researcher able to guide students as they explore the bounds of the teacher's knowledge. Such individuals may be difficult to locate. Importantly this description excludes the professional academic - who in many ways forms the lifeblood of the university system. (pp. 3-4)

    However, it is in just such broad discipline areas (most noticeably medicine) that PBL has made its greatest impact. The high value placed on experience may support a teacher-centered model of education in which the teacher embodies experience which is transmitted to students, or it may equally support a constructivist model which encourages the exposure to such experience by students. Much of this commentary on discipline-based inherent propensity for transmissive pedagogies is likely to be attributable to historical and political forces, and may more productively be associated with inherent difficulties brought about by the more general process of change itself.

    O'Hanlon, Winefield, Hejka & Chur-Hansen²¹ report on student evaluations of a first-year medical program run in parallel PBL and traditional modes, and note that although students were highly motivated by the PBL approach, overall the traditional approach was favored (found "more beneficial"). Many students were undecided. Occurring as a first-year program it is likely that one influencing factor on these results was student inability to acclimatize to the PBL philosophy, which is fundamentally different from the individualistic and competitive approaches typically rewarded during secondary schooling. This current paper identifies in this an opportunity for improving the situation with appropriate scaffolding to lessen the severity of the impact associated with any dramatic change. That is, change can be managed in order to reduce the severity of its impact; this represents one aspect of the need for scaffolding within PBL.

Issues of Performance and Traditional Assessment Instruments

    One of the focus questions of the literature review by Albanese and Mitchell¹ concerned the performance of PBL medical students in basic science exams such as the NBME I (the acronym refers to the National Board of Medical Examiners) and the USMLE I (the United States Medical Licensure Examination). They refer to a number of studies which suggest that PBL students may exhibit lower performance in such exams compared to students from conventional courses; it is important that they note that this is not a universal finding, and suggest that one explanation for the varying results may be due to different approaches to the delivery of PBL.

    In a PBL course that integrated pharmacology into 1st and 2nd year medicine,²⁴ it was found that performance of PBL students in the traditional evaluation system was favorable, despite the concerns expressed elsewhere. Two of three groups studied performed in the NBME I with slightly lower means than the national average (for pharmacology) while the remaining group equaled the national average in the USMLE I. The latest three groups equaled and bettered the USMLE I national average. They note, however, that the USMLE I examination often integrates questions across discipline boundaries, and raise the question as to whether or not this offers an advantage to PBL students. McGregor and colleagues¹⁹ were unable to register differences between didactic and PBL modes of learning in subject-based performance tests of medical students in their third-year clerkship. Lieux²⁵ resents the results of a study of PBL and conventional stream students in a food and nutrition course, and finds that they do equally well in exams, although PBL students exhibited a significantly higher attendance rate.

Inherent Demands Associated with Valued Learning

    It is reasonable to suppose that more valued learning may involve greater demands on student learning processes. Lieux²⁵ notes that student surveys of parallel PBL versus conventional-stream modes of learning show that PBL students see themselves as working harder. Perkins²⁶ examines three broad ways in which constructivism makes strong demands on learners:

1. Cognitive complexity is a feature of constructivist learning environments, in which authenticity is sought. This is particularly the case where pre-existing knowledge may be naive (such as is common in science). Perturbation is a constructivist device that exposes these misconceptions, and forces the learner to confront them. It may be likened to an induced anxiety in which the testing of concepts against reality exposes significant (and unexpected) shortcomings in the existing concepts and intuitions. Thus, the resulting cognitive load may be high. However, if this is the cost of non-avoidance of misconceptions, then it is a cost that may be regarded as worthwhile. Perkins suggests that a middle ground exists (which he refers to as the "conflict deferred" approach), where learners may postpone dealing with misconceptions until some grounding in the new domain is achieved. This is essentially an argument for early scaffolding, although it is not presented as such. Clearly, regardless of cognitive cost concerns, the "conflict buried" approach which allows misconceptions to continue is not an option if meaningful learning is desired; such an approach may be permitted in non-constructivist environments in which assessment and presentation of content reward shallow, reproductive learning strategies.
2. Task management is increased in constructivist settings, as a necessary condition for students to exhibit learning independence and a function of the emphasis on student-centerdness. The danger is in not permitting time for gradual development of these essential managerial skills, something that would be magnified in the presence of cognitive complexity. This is where constructivist techniques such as "cognitive apprenticeship" may be useful in reducing the load. Savery and Duffy⁴ identify cognitive apprenticeship as a natural device for assisting in metacognitive processes in PBL environments. That is, early scaffolding of skills essential to the constructivist process can benefit the functioning of those processes. Peterson²⁷ establishes a number of important skills that enhance PBL. Their ability to offer such enhancement suggests the importance of developing them, rather than simply expecting them to grow in response to need.
3. "Buying in" to the PBL approach is another area of possible difficulty. Students may not identify with the reasons given for not simply being told the information which teachers want them to know. That they are asked to adopt such a vastly different paradigm of learning to that in which they typically feel comfortable at the same time as they are asked to learn new material adds up to a potentially significant set of obstacles. These difficulties alter the way in which the motivational benefits associated with PBL are perceived. It is not apparent how scaffolding can bridge such perceptions, and it may be argued that the change required of the culture of student learning is so fundamentally different as to be incapable of full appreciation from within the instructivist, transmissive world-view. If such a proposition is valid, then the motivational benefits of PBL are not simply a desirable side effect, but essential to its success. If the importance of meaningful approaches to learning is obscured by preconceptions of education then high levels of motivation offered by PBL approaches may be an essential bridge towards new learning habits. One may be moved to suggest that in this context motivation itself is a form of scaffolding that is fundamental to the success of PBL.

The Effect of Tutors on PBL Success

    The last section presented some reasons for scaffolding. The tutor may be recognized as its most evident source. Zimitat and colleagues¹⁸ report that student surveys revealed that 70% of students in a PBL course found the tutors' roles essential to the success of the method.

    The desired role of the tutor in PBL must be quite different in nature to that of a tutor in a didactic system. Jones, Donnelly, Nash, Young and Schwartz²⁸ refer to two essential roles of the PBL tutor: facilitation of the learning process via prompting, and assisting in group processes to ensure that they maintain focus. They also mention that tutor training includes refraining from assuming the role of an authoritative source of knowledge. It is not apparent that the tutor's desirable relationship to subject content is well-understood; this is an issue which is relevant to a pedagogy such as PBL which values student-centered approaches to learning, thus bringing the role of the teacher's relationship to knowledge into question. Ambury²⁹ refers to tutor subject-based expertise as a disadvantage to student-centered learning and independence. However, Eagle, Harasym and Mandin³⁰ find that tutor expertise favors greater congruence between learning issues and case objectives, and stimulates greater numbers of learning issues to be explored. Sobral,¹³ in comparing staff-tutored groups with peer-tutored groups in a PBL medicine course at the University of Brasilia, witnessed no significant difference between them in terms of problem solving or student self-evaluation of skills. However, scores for meaningfulness of learning and usefulness of group work were found to be significantly higher in peer-tutored groups. The level of tutor expertise is clearly one area that would benefit from continued research.

    The importance of the tutor (without specification as to whether or not that role is filled by staff or advanced peer) is emphasized by Gijselaers and Schmidt.³¹ They found a causal relationship between tutor involvement in PBL and group processes, which in turn affects student motivation towards learning. The importance of such motivation to the success of PBL has already been discussed.

    Other correlations have been proposed. It is widely recognized that the tutor role in PBL (or more widely, constructivist) learning environments should be one of facilitation rather than instruction.¹⁴ More widely, Kember and Gow¹⁴ have found correlations between staff attitudes to teaching roles (facilitatory vs. instructional) and student approaches to learning (deep vs. surface):

...in departments where the predominant orientation is toward knowledge transmission, the students' use of deep approach is likely to decline through the period of the course of study. On the other hand, departments with a propensity toward learning facilitation tend to discourage the use of surface approaches. The effect of an orientation toward learning facilitation is perhaps more likely to manifest itself in this way rather than as a positive advance in the use of a deep approach because of the difficulty of inducing students with a propensity toward a surface approach to consistently adopt a deep approach. (pp.67-8)

    Thus, there is a relationship between teaching role and quality of learning adopted by students. It may be suggested that the facilitatory role of the tutor in PBL is essentially one of providing scaffolding, and that the relationships that have been attributed to tutors may possibly be attributable to appropriate use of scaffolding in general, with the implication that this may extrapolate to other forms of scaffolding. This may be a valuable area of further research.

    Clearly, if the effect of tutors play such an important role in the success of PBL, it is important to recognize that a mechanism for feedback regarding tutor performance is desirable. Dolmans, Wolfhagen and Snell-Balendong³² developed an evaluation form to allow formative assessment of tutor performance by students. It focuses on three areas: guidance of students through the learning process (6 questions), content knowledge input (4 questions), and commitment to the group's learning (3 questions). These areas were validated by a confirmatory factor analysis. Questions were responded to simply by indicating on a three-point Likert scale whether the result was judged to be insufficient, neutral, or sufficient. The experiences with such evaluations at the Medical School at the University of Limburg have been rated positive, and has resulted in a increase in teaching focus.

Attributes of PBL in Relation to Scaffolding Needs

    It is clear that the effect of the tutor is an important one in PBL, and that the tutor is well placed to provide scaffolding to learners. Here, the discussion turns to consider scaffolding in terms of the main attributes of PBL.

Student Control

    It is important to note that PBL's emphasis on student independence and learner ownership of problems does not preclude the use of scaffolding to assist in the development of these attributes and other skills required to enable meaningful learning. PBL lays the responsibility and, ultimately, the control of the learning process essentially at the feet of the student; however, this is not a process of abandonment. Koschmann and colleagues⁶ state that "there is a well-defined curricular infrastructure under girding the process. Cases are developed from actual patient records and are presented in the way actual patients present for care: as an ill-structured problem that must be built by inquiring to gather necessary information..." (p.242).

Authenticity

    Authenticity requires a level of "fidelity" to ensure that learning occurs in sufficiently real-world context, and also avoidance of the provision of information that a real-world context would not provide.⁶

    Honebein, Duffy and Fishman³³ identify a number of elements, which lend authenticity to a task:

• Learner ownership: This is supported by the argument that metacognition is essential to function well in complex environments and therefore they must be supported in developing a sense of responsibility for their management of problem-solving tasks, which suggests problems ownership.
• Project-based nature: This suggests a holistic representation of the task, with opportunities for authentic global (wider context) entities as well as more localized ones.
• Multiple perspectives: The empowerment of students to consider multiple perspectives when examining a problem domain is an important mechanism for developing expertise. One means for encouraging this is in the use of collaborative learning environments, as are typically used in PBL programs.

    Problem authenticity is a feature of constructivist approaches to education, and is a readily recognized aspect of PBL, as shown by the above criteria.

    Jayawickramarajah³⁴ examined the nature of six PBL problem documentation sets (via a process of content analysis) over three medical schools that revealed a number of factors influencing the construction of problem fidelity:

• Presentation format may vary from patient simulations and video recordings of cases (high fidelity) to "written simulations" in which the case is described on paper (exhibiting low fidelity).
• Manifestation may be ill structured and given as an individual variant on the range of possibilities (high fidelity) or may be a comprehensive list of the pool of manifestation possibilities (low fidelity, textbook-like).
• Associated problems may be included which complicate the scenario with possibly unrelated factors such as symptoms produced by medication or behavioral problems (high fidelity); alternatively, the problem may be filtered to present an isolated set of conditions (low fidelity).
• Context may resemble that of actual situations such as those that afford the opportunity to talk to family members of the patient (high fidelity), or may be more contrived (low fidelity).
• Objectives may be defined in terms of the health of the patient (high fidelity and integration across disciplines) or may be discipline-bound (low fidelity and artificial).

    He found that low-fidelity problems such as those stylistically borrowed from textbook models of problems to be "counterproductive in terms of the objectives of PBL. In this context textbook derived health problems are not considered very appropriate" (p. 281).

Group Processes

    A study by Tipping, Freeman and Rachlis³⁵ reveals that observed group dynamics don't necessarily match those, which are reported by staff or students:

Data collected from the observations and videotapes were markedly different from the self-reported data obtained from the students and faculty. The observers noted patterns of interaction and involvement, such as some students not participating at all for the full two hours, communication directed mostly toward the tutor and not among the group, one member sleeping during the tutorial, and a group in which the sole female member was relegated to a secretarial role. No cohesion was evident in these groups. Several aspects of productivity were not addressed. Goals were not articulated, methods for achieving goals were unclear, measurement of achievement was non-existent, and no time was spent in planning for future sessions. There was no evidence of reflection on any aspect of group behavior. (p.1052)

    Students later claimed that reasons for this difference between observed and reported behaviors included the intrusive effect of the video (although video was only one means of observation, and doesn't appear to be a mechanism that would permit some of the behavior more easily guarded against, such as sleeping!); lack of certainty as to expected roles (which is akin to suggesting that the group dynamics didn't work because they didn't work!); and lack of tutor guidance. It is suggested here that the last one is the only one with any plausibility, and emphasizes the importance of the role of the tutor in PBL. It is, most emphatically, a request for greater scaffolding. The authors suggest reasons of minimal prior exposure to group dynamics (especially as individual competitiveness is a characteristic of premedical curricula) and lack of formal training in this area; it was further suggested that students lacked a model of group effectiveness. Tutors, despite some training in the area, did not facilitate reflection on group processes, leading to the conclusion that even greater emphasis needs directing at this area in future staff training programs. Such failures in group processes may be interpreted as failures in the provision of appropriate scaffolding in both tutor facilitation of group processes and in establishing a successful mental model of interaction within the group.

    While on the topic of perceptual incongruities, Lieux²⁵ mentions the results of student course evaluations that revealed that PBL students perceived themselves as learning less content than lecture-stream students, and, conversely, lecture-stream students regarded themselves as learning more; this is in spite of the equality of performance even in traditional assessment schemes. Certainly, despite some implementations of PBL leaning towards discovery learning, content is deliverable within a PBL approach .³⁶ In addition to learning content, however, PBL students regarded themselves as learning problem-solving, communication skills, and developing a sense of "personal responsibility"; these were not found in the feedback from the lecture-based students. Similar results have occurred in the author's experiences.^20,37,38

    St-Arnaud³⁹ as cited in Hebert and Bravo⁴⁰ describes four directions in group processes:

1. Production is the contribution of individuals towards a fixed group goal, and may take the form of critiques, proposals, summaries, etc.
2. Solidarity is individual contribution towards group communication, and involves supportiveness, attention to (and recognition of) the contributions of others, etc.
3. Maintenance involves the addressing of overall group cohesiveness, and involves clarification, organization, and facilitation of individual interactions within the group.
4. Residual processes are those negative energies that are typically recognized by the symptoms of absenteeism, separate and unrelated work, etc. Group functionality lies in the ability to dwarf residual processes by those more positive energies above.

    If the group is to work effectively, some effort must be directed towards it, and facets of group dynamics such as those described need to be given recognition within the course. Hebert and Bravo⁴⁰ have used these processes in developing an instrument for measuring all facets of PBL behavior. Tipping and colleagues³⁵ recognize a number of aspects of group dynamics in their Group Observation Guide: physical and emotional climate, individual involvement, interaction, cohesion, productivity and leadership. Their study reveals that both students and staff were largely limited in their reporting of group processes to climate, interaction, productivity, and leadership, with varying levels of detail. Involvement and cohesion were unmentioned.

Discussion

    The concerns regarding performance of PBL students in traditionally-valued assessment modes is matched, as we have seen, by results which indicate comparable performance. The variation in these results becomes as interesting as the initial question. Robbs and Merideth⁷ refer to a number of "myths" which have surrounded PBL. Their observations at the University of Southern Illinois School of Medicine are that PBL students do as well in the USMLE I exams as conventional students. They also refer to PBL as having a hidden structure; perhaps it is the case that this structure is absent in some PBL programs. Albanese and Mitchell¹ also raise the matter of different forms of expression of PBL in explaining these variations with respect to such exams. It remains a matter for further research to investigate the possibility of correlations between structure and success in these exams, and what possibilities exist for establishing such scaffolding without threatening the essential nature of PBL. Phelan, Jackson and Berner⁴¹ also report no statistical differences between clinical scores for PBL versus traditional students. Furthermore, they note that this confirms findings of other recent studies, and state that it is primarily the early studies that found this to be a possible problem. Is it possible that the results are reflecting a maturation of PBL in progress? One possible scenario is that educators are discovering the delicate balance between appropriate scaffolding within constructivist pedagogues.

    Student surveys present one possible means of exploring the issues. O'Hanlon and colleagues²¹ used open questions with students engaged in a first-year PBL program. The questions and all answers with double-figure percentages (only) are reproduced here (n = 119):

1. "What were the best aspects of this module?"

• Working in a group or as a team (27.7%);
• Independent learning; active participation; research (19.3%);
• The presentation; public speaking (16.0%); and
• Case studies; clinical applications (10.1%).

    These results reflect the advantages commonly associated with the PBL approach. Interestingly, (as this is a first-year course) the transition to group work does not seem to have been problematic. The favorable impression of group work that is consistently reported is interesting in light of Sobral's¹³ findings that there is a statistically significant positive correlation between PBL scores for meaningfulness of learning and both group-work and self-evaluation. Similarly, motivation was shown to correlate positively with problem-solving scores.

2. "How could this module be improved?"

• More time; less preparation (34.5%);
• More guidance; clearer instructions and expectations (19.3); and
• Give background information (12.6%).

    There are essentially two factors here - time demands and the need for scaffolding.

3. "Do you have any comments on the general use of a problem-based approach to teaching medical knowledge?"

• Time consuming (29.4%); and
• More guidance required (23.5%).

    Once again the emphasis on scaffolding, and time is evident. Of course it is difficult to extract a sense of the extent to which this is an expression of the resistance to change (recalling that this is a first-year medical PBL program).

    Cawley⁴² produces a summary of results for PBL survey student evaluation in a mechanical engineering course (as a final-year option) on a 5-point Likert scale over 3 years, with sample sizes ranging from 21 to 37. The stable results (i.e., those which maintain their result over the years) indicate positive results for a number of factors, scoring well for interest (mean of approx. 4.6), enjoyment (mean approx. 4.3), relevance due to authenticity (approx. mean of 4.5), and an understanding of technical knowledge equal to that expected from a conventional approach (mean of approx. 4.0).

    Trends indicated through the results over the years are increasing feelings of quantity of knowledge being equal to conventional approaches as the course progresses (progressive means of 3.2, 3.2, and 3.8), and an increase in time required of PBL in comparison with lectures (means of 3.2, 4.2, and 4.4).

    With respect to the time required, the first result of 3.2 (indicating that it takes about as much time as the conventional course) is discussed in terms of the time being greater than student estimations, since (p. 183) "students could frequently be heard discussing the problems in the coffee room, an almost unknown occurrence on conventional courses, and this valuable peer learning may not have appeared in their estimates". The perceived rise in time demands of PBL in the 2nd and 3rd years of the course is proposed as possibly due to increased coursework requirements in other courses, and accompanied by staff perceptions of less time spent on PBL projects in those years.

    Although it may not be immediately apparent, one means of support of the student in the presence of perceptions of increased workload may be via alterations to the assessment scheme, offering greater recognition of the work performed. Zimitat and colleagues¹⁸ found that over half of students surveyed in a PBL course (n=231), and of whom 70% were motivated by the approach, expressed the opinion that the assessment weight for PBL cases should be increased due to the perceived extra workload in PBL.

    Other mechanisms of support through the assessment system are possible. Ma¹⁵ employed a negotiated assessment program in a course on database systems (of course, staff are essential elements in the negotiation!).

    The change to a PBL curriculum may inherently suggest redesign of assessment. Sokas and colleagues⁴³ suggest that PBL may have benefits that are difficult to quantify, and note that in response to this supposition some PBL programs have opted for modification of testing towards reduced emphasis on recall, or have been satisfied with lower scores on traditional testing methods such that they are not significantly worse than conventional teaching approaches would generate.

    It is important to cast PBL in terms of its original motivations, and note that it does not explicitly attempt to satisfy assessment schemes which reward shallow learning approaches. Additionally, PBL implies wide-ranging changes to the values of traditional education, and cannot be realistically applied "on top of" existing infrastructure. Ambury²⁹ refers to PBL in medicine as part of a larger move away from "basic sciences and organ systems as the organizing principle for the curriculum" (p.1). None of this, as has been seen, necessitates a disregard for factual information. Pereira and colleagues¹⁷ studied student evaluations of PBL medical programs at the University of the West Indies, and found that only 4 percent were dissatisfied with their knowledge base obtained via PBL learning modes. DesMarchais and Vu³ state that a move to emphasize analytic and problem-solving skills in areas of (medical) basic and clinical science using small groups increases a need for assessment. The issue of accountability becomes an important one. With respect to preclinical skills reproduction of facts is still valued and is measured by multiple-choice questions, and short-answer questions with a content-based focus. The problem-solving and analytic skills are tested with newly-developed instruments in the form of problem-analysis questions and by oral exam. Small-group competencies (such as self-directed learning, etc) are tutor-evaluated by use of a form completed after each session. Initial (staff/student) concern over the subjectivity of the tutor-based evaluation instrument subsided once it was established that it was able to identify students with problems. It was found that tutor-based assessment shows most discrimination in the presence of long-term problems and when mean ratings are sufficiently high to encourage tutors to use low rating scores for weaker students. In the absence of a high mean all scores tend to be clustered rather than exhibit this (desirable) bimodality.

    Sivam and colleagues²⁴ state that subjective attributes associated with PBL modes of learning may be difficult to measure objectively. The question as to the continued desirability of attempts at assessment objectivity remains an open one. Savery and Duffy¹⁴ use peer-evaluation and self-assessment as the only form of assessment in the medical school, with focus on self-directed learning, problem-solving, and group dynamics. This is placed in wider context where external (Medical Board) exams must still be passed at the end of second year. Other forms of assessment focus on what the learners identify as valuable.

    Hebert and Bravo⁴⁰ developed a tutorial-based test for PBL students (the Tutotest). The test uses 4-point Likert scales over 44 questions to allow tutors evaluate their students. A trial of the test (n=270) incorporated a factor analysis which revealed four factors (accounting for 82% of the variance) which were described as

1. group effectiveness - 61% of the variance;
2. communication and leadership - 11% of variance;
3. scientific curiosity - 6% of variance; and
4. respect for colleagues - 4% of variance.

    The factor analysis was unable to support their initial position that effective use of PBL and effective group communication were separately identifiable, and indicated existence within the one factor. They retrospectively identify this as a natural association, due to the group-based nature of PBL. Here, it strengthens the argument for emphasizing the group processes in PBL environments as a factor for increased successful usage of the PBL style (i.e., "getting with the program" requires feeling comfortable with small group dynamics). This inevitably carries through to the tutor, given that 1) group dynamics are important 2) the tutor has facilitation of group dynamics as one of the tasks associated with the job, and 3) the (previously indicated) importance that students give to the role of the tutor in success of the approach.

    Des Marchais and Vu ³ state that:

A student's learning evaluation system has so great an effect on the way students dedicate their energy in learning medicine that any institution must pay great attention to developing an evaluation congruent with its curriculum philosophy, its educational goals, and its students' learning needs. (p. 282)

    They illustrate the development of such a scheme that has been found to successfully meet these demands.

    O'Hanlon and colleagues²¹ stress an important aspect of the evaluation of PBL programs that is under-represented in the literature, and that is the implications of PBL modes of learning on students with a non-English background and from a cultural perspective where the nature of learning is likely to consist of different values. They report that an optional supplementary tutorial to offer support for these learners. Those who attended rated the supplementary tutorial highly (76.7% found it helpful) and they indicated higher responses to the overall favoring of PBL on the general survey than those who chose not to attend. Furthermore, those higher responses were significant statistically for questions relating to "guidance in group work" and "spread of work". This is a good indication of the value of scaffolding in this area. Clearly, more research should be directed at ESL and cultural issues, given the cultural mix of students and the prevalence of PBL modes of learning.

Conclusion

    In changing from a subject-based discipline to an integrated PBL mode, it is often difficult to anticipate the need for scaffolding in non-discipline areas (such as group dynamics, metacognition, etc.). The assumption of some unspecified, automatic development of these processes may be at the heart of some results, which we have seen where PBL improves over time, both academically and in terms of student perception. It may be a measure of the difficulty associated with transition from subject-based learning approaches to PBL, in fear that less content may be covered if too much time is devoted to group processes, etc. It may need to be accepted that PBL involves a slower start-up in terms of the discipline-based content that is covered. This may be due to the development of important "hidden" skills, which will ultimately facilitate deep approaches to learning. This is an important time in which to ensure that adequate scaffolding exists to support this type of development. That such generic skills are important is widely recognized, as may be seen in the following quote by Koschmann and colleagues:⁶

In summary, educating competent physicians is difficult. The amount of material that must be learned is vast, and is conceptually complex. Patient problems are ill-structured; information about the problem must be uncovered, dynamically changing the problem as it emerges, and decisions must be made and action taken in the absence of complete information. Finally, medical knowledge and techniques for acquiring patient information are changing at a phenomenal rate, but little is done in traditional medical education to develop the cognitive disposition and competence necessary for the student to become an effective lifelong learner. (p. 240)

    One of the difficulties in measuring learning outcomes is that PBL advocates frequently claim that the differences between it and conventional, subject-based learning approaches occur principally at deep learning levels, and many conventional evaluation processes measure this inadequately.¹ It is not always clear that assessment is matched to PBL modes of learning, which may be a hidden source of anxiety for many students, who may recognize the assessment criteria as "shallow", but not feel sufficiently acquainted with them (through lack of practice). Morrison and Murray,⁴⁴ in discussing the results of a student survey of PBL course which provided evidence of high motivation but low relevance, suggest that at least some of basis for the low scores in perceived relevance could be directly related to exam anxiety, stating that

By the time the students enter the final rotation of their medical course they are already preoccupied with passing their final examinations. As their course so far has tended to encourage superficial learning likely to aid in passing examinations, the problem-based learning, which was felt to be time-consuming although promoting deep learning .... was judged to be less relevant. (p.145)

    Support structures need to consider the wider context in which students find themselves. The benefits of PBL are often constrained by implementation factors, such as consideration for external (non-PBL) courses and student perceptions of time requirements, or by gradual introduction of the PBL environment (such as gradated fidelity), or by bounding the degree to which ideals are attempted to be implemented. This may mean temporary departure from some of the principles in order to introduce them in a way that is meaningful to the student. For example, excessive adherence to the belief in student-centerdness could lead to a position where students react against the learning environment as a whole. Such biases are not necessarily inherent to the PBL philosophy, but rather more likely to be an implementational over-correction in adopting PBL for the first time. Some studies indicate that student reaction to PBL becomes more favorable in later incarnations of the course, and this may possibly be a result of incorporating pragmatism into perhaps an initially overly-idealized curriculum.

    Drinan⁴⁵ suggests that PBL demands a level of maturity that may not be applicable to all undergraduate scenarios, although it is appropriate to postgraduate expectations. Others report similar feelings with different aspects of the PBL environment, such as in group dynamics.³⁵ Clearly, PBL requires certain skills in order to be optimally effective. These skills should not be assumed. They require a process of development in order to empower the student with the ability to realize the ideals of PBL. It does not weaken the case for PBL to actively address the learning of such skills, but rather is likely to strengthen the success of the approach.

    Related to this is the importance of not assuming that "peripheral" aspects such as logistics, politics, and staff concepts about learning are secondary. In many ways they become crucial to the success of a PBL program, due to the ability for change to result in upheaval, and the fact that they are capable of being limiting factors despite their non-pedagogical position in the scheme of things.

    Schilling, Ginn, Mickelson and Roth⁴⁶ discuss a need to change the nature of library services (and librarian training) in response to the different emphasis suggested by PBL programs. This reaction to PBL is a strong suggestion of the support structure that PBL may generate, and indirectly indicates the need for a broad approach to scaffolding in PBL. Other research confirms that at least some of the excess time that students associate with PBL may be directed at information gathering, and therefore suggests that it is the nature of a problem that can be fixed⁴⁴ "in a packed medical curriculum the advantages of ease of retrieval of information from textbooks and the compact presentation of facts possible in lectures are all too readily appreciated" (p.144).

    Finally, it is suggested that PBL may operate as its own support mechanism, by offering it in early years in a more structured format than its ideals suggest, so that the desired skills may be accommodated and enable a more mature approach to senior courses offered in PBL style.

Acknowledgments

The author is grateful for the important feedback on this paper given by Ken Appleton. Comments made by the MEO reviewers were also useful in bringing this paper to its final form.

References

1. Albanese MA, Mitchell S. Problem-Based Learning: a review of literature on its outcomes and implementation issues. Academic Medicine 1993;68(1):52-81.
2. Feltovich PJ, Spiro RJ, Coulson RL. The nature of conceptual understanding in biomedicine: the deep structure of complex ideas and the development of misconceptions. In: Evans D, Patel V, editors. Cognitive science in medicine: Biomedical modeling. Cambridge, MA: MIT Press, 1989:111-172.
3. DesMarchais JE, Vu NV. Designing and evaluating the student assessment system in the preclinincal problem-based curriculum at Sherbrooke. Academic Medicine 1996;71(3):274-283.
4. Savery JR, & Duffy TM. Problem Based Learning: an instructional model and its constructivist framework. Educational Technology 1995;35(5):31-38.
5. Schmidt HG. Problem-Based Learning: rational and description. Medical Education 1983;17:11-16.
6. Koschmann TD, Myers AC, Feltovich PJ, Barrows BS. Using technology to assist in realizing effective learning and instruction: a principled approach to the use of computers in collaborative learning. Journal of the Learning Sciences 1994;3(3):227-264.
7. Robbs J, Meredith S. The Problem-Based Learning curriculum at Southern Illinois University School of Medicine [online] 1994. Available from: URL http://www.suimed.edu/pblc/pblcur.html (link nolonger active)
8. Marton F, Saljo R. On qualitative differences in learning, outcome, and process (I). British Journal of Educational Psychology 1976;46:4-11.
9. Gibbs G, Morgan A, Taylor E. A review of the research of Ference Marton and the Goteborg Research Group: a phenomenological research perspective on learning. Higher Education 1982;11:123-145.
10. Coles CR. Differences between conventional and problem-based curricula in their students' approaches to studying. Medical Education 1985;19:308-309.
11. Newble DI, Clarke RM. The approaches to learning of students in a traditional and in an innovative problem-based medical school. Academic Medicine 1986;67:557-565.
12. Gibbs G. Improving the Quality of Learning. Technical and Educational Services: Bristol, 1992.
13. Sobral DT. Peer tutoring and student outcomes in a problem-based course. Medical Education 1995;27:284-289.
14. Kember D, Gow L. Orientations to teaching and their effect on the quality of learning. Journal of Higher Education 1994;65(1):58-74.
15. Ma J. Problem-Based Learning with database systems. Computers and Education 1994;22(3):257-263.
16. Biggs JB. Student Approaches to Learning. Australian Council for Education Research: Melbourne, 1987.
17. Pereira LMP, Telang BV, Butler KA. Preliminary evaluation of a new curriculum - incorporation of Problem Based Learning (PBL) into the traditional format. Medical Teacher 1993;15(4):351-364.
18. Zimitat C, Hamilton S, DeJersey J, Reilly P, Ward L. Problem-Based Learning in Metabolic Biochemistry [online] 1994. Available from: URL http://florey.biosci.uq.edu.au/BiochemEd/PBLmetab.htm
19. McGregor DB, Arcomano TR, Bjerke HS, Little AG. (1995). Problem orientation is a new approach to surgical education. The American Journal of Surgery 1995;170:656-659.
20. Novae Research Group. An Evaluation of the 1996 Problem-Based Learning Course. Unpublished report to the Basser Department of Computer Science, University of Sydney; 1996.
21. O'Hanlon A, Winefield H, Hejka E, Chur-Hansen A. Initial responses of first-year medical students to Problem-Based Learning in a behavioural science course: role of language background and course content. Medical Education 1995;29:198-204.
22. Camp G. Problem-Based Learning: a paradigm shift or a passing fad? Medical Education Online [serial online] 1996;1,2. Available from: URL http://www.utmb.edu/meo/f0000003.pdf
23. Kenley R. Problem Based Learning: within a traditional teaching environment [online] 1995. Available from: URL http://www.arbld.unimelb.edu.au/~kenley/conf/papers/rk_a_p1.htm
24. Sivam SP, Iatridis PG, Vaughn S. Integration of pharmacology into a Problem-Based Learning curriculum for medical students. Medical Education 1995;29:289-296.
25. Lieux EM. A comparative study of learning in lecture vs. problem-based format [online] 1996. Available from: URL http://www.physics.udel.edu/wwwusers/pbl/cte/spr96-nutr.html
26. Perkins DN. What constructivism demands of the learner. Educational Technology 1991 Sept;19-21.
27. Peterson M. Skills to enhance Problem-Based Learning. Medical Education Online [serial online] 1997;2,3. Available from: URL http://www.utmb.edu/meo/f0000009.pdf
28. Jones RO, Donnelly MB, Nash PP, Young B, Schwartz RW. The ongoing development of a problem-based surgery clearkship: year three. Medical Teacher 1993;15(2/3):207-215.
29. Ambury G. Beginning to tutor Problem-Based Learning: a qualitative investigation of adragogy in medical curriculum innovation [online] 1995. Available from: URL http://educ.queensu.ca/~amburyg/pbl-c.html
30. Eagle CJ, Harasym PH, Mandin H. Effects of tutors with case expertise on Problem-Based Learning issues. Academic Medicine 1992;67(7):465-469.
31. Gijselaers WH, Schmidt HG. Development and evaluation of a causal model of Problem-Based Learning. In: Nooman AM, Schmidt HG, Ezzat ES, editors. Innovation in Medical Education: an evaluation of its present status. Springer-Verlag: Berlin; 1990:95-133.
32. Dolmans DJJM, Wolfhagen IHAP, Snellen-Balendong HAM. Improving the effectiveness of tutors in Problem-Based Learning. Medical Teacher 1994; 16(4):369-377.
33. Honebein PC, Duffy TM, Fishman BJ. Constructivism and the design of learning environments: context and authentic activities for learning. In: Duffy TM, Lowyck J, Jonassen DH, editors. Designing Environments for Constructivist Learning, Springer-Verlag: Berlin; 1991:87-108.
34. Jayawickramarajah PT. Problems for Problem-Based Learning: a comparative study of documents. Medical Education, 1996;30:272-282.
35. Tipping J, Freeman RF, Rachlis AR. Using faculty and student perceptions of group dynamics to develop recommendations for PBL training. Academic Medicine 1995;70(11):1050-1052.
36. Fekete A, Greening T, Kingston J. Conveying technical content in a curriculum using Problem-Based Learning. Proceedings of the Third Australasian Conference of Computer Science Education. ACM Press: Brisbane; 1998.
37. Greening T, Kay J, Kingston JH, Crawford K. Problem-Based Learning of first-year computer science. Proceedings of the First Australasian Conference of Computer Science Education. ACM Press: Sydney; 1996.
38. Greening T, Kay J, Kingston JH, Crawford K. Results of a PBL trial in first-year computer science. Proceedings of the Second Australasian Conference of Computer Science Education. ACM Press: Melbourne; 1997.
39. St-Arnaud Y. Les Petits Groupes. Editions du CIM: Quebec; 1978.
40. Hebert R, Bravo G. Development and validation of an evaluation instrument for medical students in tutorials. Academic Medicine, 1996;71(5):488-494.
41. Phelan ST, Jackson JR, Berner ES. Comparison of problem-based and traditional education on student performance in the obstetrics and gynecology clerkship. Obstetrics and Gynecology 1993;82(1):159-161.
42. Cawley P. A problem-based module in mechanical engineering. In Boud D, Feletti G, editors. The Challenge of Problem-Based Learning.Kogan Page: London; 1991:177-185.
43. Sokas RK, Fenton J, Foran D, Diserens D, Bargmann E, Schwartz W, Simmens S. Environmental medicine: its introduction into a medical school primary care requirement. Medical Education 1993;27: 410-415.
44. Morrison JM, Murray TS. An experiment in Problem-Based Learning. Medical Education 1994;28:139-145.
45. Drinan J. The limits of Problem-Based Learning. In: Boud D, Feletti G, editors. The Challenge of Problem-Based Learning.Kogan Page: London; 1991:315-321.
46. Schilling K, Ginn DS, Mickelson P, Roth LH. Integration of information-seeking skills and activities into a problem-based curriculum. Bulletin of the Medical Librarian Association 1995;83(2):176-183.

Tony Greening is a lecturer at the School of Information Technology and Mathematical Sciences, University of Ballarat, Australia. He can be reached by e-mail at t.greening@ballarat.edu.au.

Greening, T. Scaffolding for success in PBL. Med Educ Online [serial online] 1998;3,4. Available . from URL http://www.Med-Ed-Online.org.