Policy Analysis in Medical Education: A Structured Approach*David W. Musick, Ph.D.University of Kentucky College of Medicine
This paper describes a twelve step, explicit approach to policy analysis within medical schools. An example of how to use this approach is included. Adoption of this method of policy will result in a more explicit and formal approach to curricular governance and educational decision-making within medical schools. "Policy analysis" is a formal discipline within the field of education; its prominence is evidenced most recently by the establishment of a special division (Division L, "Educational Policy and Politics") within the American Educational Research Association1 which is devoted to the stimulation of "informed and systematic debate, analysis, research, evaluation and recommendations concerning educational policy as well as political, legal, and fiscal matters related to education"1. Policy analysis in education is used to equip those in governance positions in secondary schools and institutions of higher learning with the "tools" they need to direct educational practice. A related purpose is to discover how (or through what process) educational policies should be formulated and adopted. As a formal discipline, however, policy analysis has not gained widespread acceptance within medical education. Medical educators should consider using the "policy sciences" to better equip educational decision-makers within individual medical schools. What is policy analysis? The search for a precise definition is difficult; the discipline has only recently been recognized as legitimate in educational circles, and suffers from "an uncertain identity and vagueness of conceptualization"2. The beginning of policy analysis as a science has been associated with the "Great Society" programs of the 1960s, where it was determined that a formal method would be needed to evaluate the effectiveness of social programs instituted by Presidents Kennedy and Johnson.3 Within education, policy analysis is described in similar terms: as a way to evaluate the effectiveness of a given educational method or program. Evaluation of an educational method, in turn, leads to the establishment of educational policies that reflect successful outcomes or approaches. Prior to the mid-1980s, policy analysis was focused on specific policy recommendations and their practical feasibility, rather than on the processes by which those policies were developed or the people involved in their development.4 Since that time, however, policy analysis has become increasingly concerned with process issues; while initially conceived as a way of evaluating system(s) at a macro (or institutional) level, it has developed over time into a discipline that is primarily concerned with analyzing the policy process itself.5 Policy analysis, then, is concerned with two distinct but related activities: the contents of a given policy and the process by which the policy was developed and/or implemented. It is in the second sense (i.e., an approach to critically examining the policy process itself) that policy analysis has gained stature as a science.5,6 As applied to the field of health care and the education of health professionals, policy analysis is still in its infancy and searching for meaning; as yet, there is no universal definition of what constitutes "policy analysis" within medical education. A proposed definition is as follows: medical education policy analysis is the field of study and practice wherein the priorities, values, resources and educational processes devoted to medical education are examined. Within medical education, some have used the tools of policy analysis to examine certain educational issues. Medical education policy analysis efforts are often, for example, carried out within the context of "curriculum evaluation" or "curricular inquiry".7-13 There is, of course, an entire discipline within education known as Curriculum Studies; this discipline is most often found within Colleges of Education at larger universities and offers formal degree programs. Much of what occurs within the discipline of curriculum studies appears to be similar to policy analysis. However, policy analysis should be viewed as a broader construct than that of curriculum studies or inquiry, which is defined as "deliberation among stakeholders who .reach a consensus on how to solve educational problems in a particular situation".14 While policy analysis techniques may be applied to curricular issues (as will be shown in this paper), such techniques are clearly not confined to those issues; policy analysis may be applied to other medical education issues (e.g., institutional mission statements or values, allocation of financial resources, governance) as well. The primary purpose of this review paper is twofold: to review the medical education and related health professions educational literature in order to determine whether and how the tools of policy analysis have been applied within medical education; and, to provide a proposed framework or approach to policy analysis which could be used by medical educators. Included is an example of how this framework may be used to improve the process of policy analysis within a given educational institution, with an emphasis on educational decision-making. Literature Review The selection of source documents is a critically important step in reviewing the literature on a given topic. In conducting this review, the author accessed a variety of databases related to health professions education. Key search terms used were "policy analysis", "educational policy analysis", and "medical education policy analysis". The time frame of the review was the fifteen year period from 1982 to 1996. The English language search included MEDLINE as the source of undergraduate and graduate medical education citations, and the ERIC and INFOTRAC databases as sources of medical education articles published in nonmedical journals. The search revealed a total of forty specific citations listing the term "medical education policy analysis" as a major subject heading (Table 1). However, there were hundreds of additional citations that were located when searching these databases using the terms "educational policy analysis" and "policy analysis". The educational literature is replete with examples of policy-oriented articles on particular educational issues. A large number of these articles are not based on research data, but rather are reflective, opinion-oriented commentaries. 15-19 Indeed, one of the noteable features of papers categorized under these search terms is that they are most often written by people in positions of professional and/or political influence; this phenomenon in medicine has been aptly labeled as "cultural authority".20 Policy-oriented papers in the health professions education literature often appear under such titles as "commentary", "opinion", "reflections", or "editorials". The topics listed in Table 1 demonstrate the range of issues which are addressed as policy matters; most will be familiar to those who are involved in the education of health care professionals. It is also apparent from reviewing the literature that the approach to policy analysis taken by the authors of these policy-oriented articles appears to be of two distinct types: opinion-oriented commentaries and more rigorous analysis of existing policy issues. Many such articles advocate the development of more rigorous theoretical approaches to medical education.16 Again, one must distinguish between the analysis of policy content and analysis of the policy process. Much of what is listed as policy analysis in the health professions literature is, in fact, descriptive in nature rather than rigorous or analytic.2 Various policy initiatives are simply described or advocated, with no effort made to examine the process by which the advocated policy was developed, who was involved, or what the intended or unintended consequences of the policy could be. Such an approach to policy formation is merely an attempt to promote the values, understandings and interests of a particular person or group.2 Synthesis Representative of the field of educational policy analysis as a distinct discipline is the work of Carol Weiss.21,22 According to Weiss, much of what passes for policy analysis and discourse is adapted from the social sciences. Concepts from the social sciences, such as professional norms, organizational culture, social networking, and bureaucratic/political structures, are "borrowed" and applied to other (e.g. educational) realms. This, in turn, helps to disguise the fact that: "much of what policy makers know comes from disparate, unorganized and inchoate experience social science concepts give form to this tacit knowledge and enable policy makers to manipulate and use it". Weiss further states that policy is the product of interplay among four essential components, the "four Is" of policy analysis: institution, information, ideology, and interests. To elaborate, policy decisions are made within given educational institutions. The process of policy formation and implementation results from a combination of rational information-processing, ideologies of various kinds, and the interests of self and/or others who may be affected by the policies. Weiss approach is useful in understanding contextual factors which affect policy development, but does not provide an overall framework for policy analysis. Short24 has described an approach to policy analysis in education which is research-oriented. He outlines ten methods of inquiry currently in use within the field of curriculum studies, and describes their use in analyzing educational research and evaluation. These ten methods are: philosophical, historical, scientific, artistic, moral, religious, interpretive, instrumental, deliberative and action-oriented. These distinctions are useful in terms of helping medical educators understand the various aspects of policy analysis and how they might relate to medical education policy. Humes4 examines key questions used by educational decision-makers in order to understand all aspects of the policy analysis and development process. Key questions are: 1) By what processes do new educational policies begin to be formed?; 2) How are these processes influenced by ideological, political, economic, and cultural concerns?; 3) Who has the most influence in the formation of new educational policies (i.e., politicians, bureaucrats, or teaching professionals within education)?; 4) Where does the responsibility lie for the success or failure of new educational policies (i.e., with policy makers, managers or teachers)? This author also stresses the importance of involving parties from both "inside" and "outside" the formal educational governing body in analyzing new educational policy. Reid23 advocates a structured approach to educational policy analysis which is geared toward curricular issues. He states that five crucial areas must be considered in developing educational/curricular policies: teachers, learners, the "educational milieu" (or environment), subject matter, and "curriculum-making" (i.e., educational decision-making processes). He laments the "intrusion of interests outside the professional world of education" into educational policy development, stating that such intrusion has led to "approaches [which] stress top-down control and short-term goals" rather than sound educational policies. Based on this review, what seems missing from the medical education literature is a rigorous methodological approach to policy analysis which can be used within the unique context of medical education. In short, a "tool" is needed which can be used by decision-makers in medical schools to assist them in the process of policy analysis. Such a tool does not appear to exist at present. A Specific Example A policy issue which has been addressed within the field of medical education in recent years demonstrates the need for an analytic approach to policy analysis. Debate has been evidenced on issues related to governance structures within medical schools.24,25 What is often a primary feature of such debate appears to involve a clash between two diverse views of authority, namely, professional authority and administrative authority. As stated by Etzioni26, "the most basic principle of administrative authority and the most basic principle of authority based on knowledge (or professional authority) not only are not identical but are quite incompatible the ultimate justification for a professional act is that it is, to the best of the professionals knowledge, the right act. He might consult his colleagues before he acts, but the decision is his. If he errs, he still will be defended by his peers. The ultimate justification of an administrative act, however, is that it is in line with the organizations rules and regulations, and that it has been approved (directly or by implication) by a superior rank the question is how to create and use knowledge without undermining the organization" (emphasis added) The question is well stated. How should a group of academic decision-makers, such as those involved in governing a medical school, approach policy issues related to medical education? How can educational governance which is based on professional knowledge occur, given the need to proceed in a fashion that will not undermine the organizations administrative authority? Governance of medical education is a basic policy issue which is being addressed by many medical schools at present, particularly as they face enormous pressures in academic medicine during a time of rapid change in the health care marketplace.27,19 Would a more systematic approach to policy analysis and development prove useful in addressing this issue? Rather than simply relying on the descriptive opinions of those in positions of influence within the field of medical education, can decision-makers develop a more analytic approach to examining many of the most pressing policy issues we face? Discussion and Implications What follows is a formal approach to policy analysis in medical education which is based primarily on the literature reviewed above and the authors formal training in higher education policy analysis. This approach has been further refined by virtue of recent experience within a medical school which has undergone an innovative curricular reform project.28 The framework presented herein is a systematic approach to policy analysis in medical education. Decision-makers may wish to use this framework as part of the process of policy development in a given medical school, at either macro (organizational) or micro (departmental or unit) levels.Each of the twelve "ingredients" of this policy analysis framework, which are the creation of Richard LaBrecque, professor emeritus, University of Kentucky College of Education, represents a different way of thinking through a specific aspect of a given policy. The process of analyzing a given educational policy within a medical school should focus on the following twelve ingredients of successful policy analysis:
It is not assumed that each of the twelve components of policy analysis would be of equal value to a given policy discussion. However, it is suggested that an examination of all twelve components be considered essential to the process of developing sound educational policy. The Use of A Structured Approach to Policy Analysis As an example of how to use this policy analysis tool in a medical education context, consider a familiar issue related to the curriculum of a medical school: medical ethics education. While public interest in ethical issues within medicine and biomedical research seems to be exploding, it is apparent that the time given to this subject in most medical schools has only recently increased.29 As a hypothetical policy analysis "case", suppose the Curriculum Committee at the "XYZ Medical School" is presented with a petition signed by one-third of the schools students wherein a request is made to increase the amount of medical ethics education within the curriculum, including a recommendation that a separate, identifiable course called "Medical Ethics" be offered. This request represents an opinion that decision-makers at "XYZ Medical School" should reexamine and perhaps change an existing educational policy. How or by what process should members of the Committee go about considering this issue? Among the steps the Curriculum Committee should take, the use of the twelve policy analysis ingredients as discussed previously should be incorporated into a formal process of policy analysis and educational decision-making. So, for example, the Committee would need to ensure that the following general types of information were available before a final decision is made on whether to increase the amount of curricular content in medical ethics:
Conclusions and Implications Within the field of policy analysis, there is presently a difference of opinion about the role of professional policy analysts. Some feel that such analysts should simply "diagnose" social, educational or political problems and clarify the issues involved. This more "neutral" approach would define policy analysis as "a search for plausible argument instead of truth"6; in this view, policy analysis should be limited to analysis of existing policy. Another position is that policy analysts should not only diagnose a given problem, but go on to recommend workable solutions to the problem.5 In this view, policy analysts are not simply expected to provide credible information about existing policy; they are also expected to be problem solvers. Some would refer to this approach as "policy development", and include the promulgation of new and/or revised policy as part of the role of the policy analyst. The intricacies of this debate aside, it seems likely that decision-makers within medical education fit into the second category rather than the first. The development of educational policy within medical schools normally involves not only "diagnosing" various problems but attempting to solve them. The Curriculum Committee of a given medical school is usually asked to identify and solve educational problems through sound policy analysis and development. The approach advocated herein for analyzing or developing educational policy within a medical school involves at least four assumptions. First, policy analysis must be viewed as a developmental process; sound policy decisions should not be made hastily or without adequate consideration of all related issues. Second, such a process must be collegial and involve key decision-makers who occupy both professional and administrative positions. Third, such a process must also involve the representation of those who will be primarily affected by the given policy. A recognition of this third assumption, for example, would mean that the Curriculum Committee of a given medical school would include at least one (or more) representative(s) of the medical student body.30 Fourth, a given policy is only effective if it is disseminated, communicated and monitored; no matter how carefully the policy development process is guided, the resulting policy cannot be effective without a mechanism for follow up review by those responsible for its implementation. This approach to policy analysis may be viewed critically by some, perhaps because it appears to be too time consuming or formal. However, it is the view of this author that the development of sound educational policies deserves a considerable investment of time. The use of the policy analysis tool described herein is an attempt to make policy development more explicit and less subject to what Lomas31 refers to as policy-making "under conditions of uncertainty". In advocating a more rigorous approach to policy analysis, it is hoped that medical educators can avoid making "decisions [which] are often byproducts of unintended and unplanned activity".32 Medical education policy makers often find themselves operating within a "policy vacuum", due to a lack of information upon which a given policy may be analyzed. The use of a policy analysis tool such as the one described could result in "policy profile" documents or summaries. These summaries, which would contain a written record of policy discussions, deliberations and decisions made by educational decision-makers, could then be made a part of the formal minutes or other records of policy development groups (e.g., academic councils, curriculum committees). Educational policy decisions in medical schools about such crucial topics as governance structures, resource allocation, curricular objectives, and graduation requirements should not be based on goal ambiguity and implicit, rarely-stated policy assumptions. The potential results of such an approach are well known, and have been summarized ingeniously by Powell33 as "Disseminated Intracurricular Fragmentation". By engaging in a more systematic approach to policy analysis, medical educators will view decision-making as a deliberate, formal process. This, in turn, will lead to more informed decisions based on explicit assumptions which are shared by the entire medical school community. Policy analysis in medical education appears to be an area that is ripe for future exploration. Research into such areas as educational decision-making processes, academic governance structures, use and control of educational resources, and teaching incentives and reward structures have recently begun to appear in the medical education literature. Such research has generally focused on the development of educational policy at individual medical schools, and how the experiences of educators at these schools may provide insight for the benefit of those at other institutions. Still, there is relatively little information available regarding how the process of educational policy development occurs within medical schools. How faculty and administrators at these schools grapple with such topics as resource allocation, academic governance, curriculum design, teaching methodologies, and program evaluation will continue to be crucially important, as medical school faculty struggle to find the necessary time to devote to the educational enterprise. For example, the development of "centralized" academic governance units, as advocated by some, should occur at a given school only if this change is based on the results of careful, systematic research and policy analysis which addresses the advantages and disadvantages of this approach.34 Even when policy analysis is conducted in a more formal, structured manner (as advocated in this paper), medical educators should continue to recognize the "human element" involved. Often, it is the people involved in policy development that make the critical difference. This human element is summarized by the experiences of some who have served in decision-making roles within academic governance: "organizations have two problems: to choose the right thing to do, and to get it done. These two tasks call for different procedures, and the requirements are often in conflict".35 By considering policy analysis and development as a formal activity, using a systematic framework which takes into account the twelve ingredients of successful policy analysis, educational decision-makers will craft more informed policy decisions. It is then necessary to carry out those well-crafted policies, or (as stated above) "to get it done". Note: The author wishes to express appreciation to Professor Richard LaBrecque, University of Kentucky College of Education, for his creative guidance and stimulating instruction in regard to the various approaches to educational policy analysis. References
Dr. Musick, is an Assistant Professor and Interim Administrator for the Department of Physical Medicine & Rehabilitation and an Assistant Professor in the Department of Anesthesiology, University of Kentucky College of Medicine. He can be reached via e-mail at dwmusi00@pop.uky.edu Musick DW. Policy analysis in medical education: A structured approach. Med Educ Online [serial online] 1998:3,2. Available from URL http://www.Med-Ed-Online. Table 1 Prominent Policy Issues in Medicial Eduation
*Resulting from Health Professions Education Literature Search using Selected Key Terms, 1982-1996
Table 1 References Albanese, M.A. and Mitchell, S. "Problem-Based Learning: A Review of Literature on Its Outcomes and Implementation Issues", Academic Medicine 1993 January; 68 (1): 52-81. Ayanian, J.Z.; Cassell, E.J.; Altman, D.; Cohen, J.; Arnstein, S.R.; Haspel, L.U. "The Prospect of Sweeping Reform in Graduate Medical Education", Milbank Quarterly 1994 Winter; 72 (4): 705-33. Berkson, L. "Problem-Based Learning: Have the Expectations Been Met?", Academic Medicine 1993 October (Supp); 68 (10): S79-S88. Bernstein, E.; Goldfrank, L.R.; Kellerman, A.L.; Hargarten, S.W.; Jui, J.; Fish, S.S.; Herbert, B.H.; Flores, C.; Caravati, M.E.; Krishel, S. "A Public Health Approach to Emergency Medicine: Preparing for the 21st Century", Academic Emergency Medicine 1994 May-June; 1 (3): 277-86. Bickel, J.L. "Medical Students Professional Ethics: Defining the Problems and Developing Resources", Academic Medicine 1991 December; 66 (12): 726-9. Blumenthal, D. and Thier, S.O. "Managed Care and Medical Education", Journal of the American Medical Association 1996 4 September; 276 (9): 725-7. Bryan, G.T. "The Role and Responsibility of the Dean in Promoting Curricular Innovation", Teaching and Learning in Medicine 1994; 6 (3): 221-3. Cohen, J.; Dannefer, E.F.; Seidel, H.M.; Weisman, C.S.; Wexler, P.; Brown, T.M.; Brieger, G.H.; Margolis, S.; Ross, L.R.; Kunitz, S.J. "Medical Education Change: A Detailed Study of Six Medical Schools", Medical Education 1994 September; 28 (5): 350-60. Cohen, J.J. and Todd, J.S. "AAMC and AMA Joint Statement on Physician Workforce Planning and GME Reform Policies", Journal of the American Medical Association 7 September 1994; 272 (9): 712. Coles, C.R. and Grant, J.G. "Curriculum Evaluation in Medical and Health Care Education", Medical Education 1985; 19: 405-22. Conn, H.L. "Assessing the Clinical Skills of Foreign Medical Graduates", Journal of Medical Education 1986 November; 61 (11): 863-71. Culver, C.M.; Clouser, K.D.; Gert, B.; Brody, H.; Fletcher, J.; Jonsen, A.; Kapelman, L.; Lynn, J.; Siegler, M.; Wikler, D. "Basic Curricular Goals in Medical Ethics", New England Journal of Medicine 1985 January 24; 312 (4): 253-6. Dimitroff, A. and Davis, W.K. "Content Analysis of Research in Undergraduate Medical Education", Academic Medicine 1996 January; 71 (1): 60-67. Dorsey, J.K. and Colliver, J.A. "Effect of Anonymous Test Grading on Passing Rates as Related to Gender and Race", Academic Medicine 1995 April; 70 (4): 321-3. Fisher, E.S. and Welch, H.G. "The Future of the Department of Veterans Affairs Health Care System", Military Medicine 1995 October; 160 (10): 651-5. Friedman, C.P.; de Bliek, R.; Greer, D.S.; Mennin, S.P.; Norman, G.R.; Sheps, C.G.; Swanson, D.B.; Woodward, C.A. "Charting the Winds of Change: Evaluating Innovative Medical Curricula", Academic Medicine 1990 January; 65 (1): 8-14. Grum, C.M.; Richard, P.J.; Woolliscroft, J.O. "Consequences of Shifting Medical Student Education to the Outpatient Setting: Effects on Performances and Experiences", Academic Medicine 1996 January (Supp); 71 (1): 995-1015. Halperin, E.C.; Byyny, R.L.; Moore, S.; Morahan, P.S. "What Medical Schools and Universities Can Learn from One Another", Academic Medicine 1995 October; 70 (10): 879-83. Hendricson, W.D.; Katz, M.S.; Hoy, L.J. "Survey on Curriculum Committees at US and Canadian Medical Schools", Journal of Medical Education 1988 October; 63: 762-74. Huppatz, C. "The Essential Role of the Student in Curriculum Planning", Medical Education 1996 January; 30 (1): 9-13. Iglehart, J.K. "Health Care Reform and Graduate Medical Education", New England Journal of Medicine 1994 April 21; 330 (16): 1167-71. Kendall, J.W. "The Evolution of American Medical Association Policies Concerning Health Care of Veterans", Military Medicine October 1995; 160 (10): 518-21. Krol, D. "The Numerology of Graduate Medical Education Reform", Journal of the American Medical Association 1994 May 4; 271 (17): 1369-70. Lenhart, S.A. and Evans, C.H. "Sexual Harassment and Gender Discrimination: A Primer for Women Physicians", Journal of the American Medical Womens Association 1991; 46 (3): 77-82. McCann, S. and Weinman, J. "Empowering the Patient in Consultation: A Pilot Study", Patient Education and Counseling 1996 April; 27 (3): 227-34. Miles, S.A.; Lane, L.W.; Bickel, J.; Walker, R.M.; Cassel, C.K. "Medical Ethics Education: Coming of Age", Academic Medicine 1989 December; 64 (12): 705-14. Nolan, J. and Cooper, R.A. "Should the Government Require US Medical Schools to Produce More Primary Health Care Providers?", CQ Researcher 1995 17 March; 5 (10): 233. Nora, L.M. "Sexual Harassment in Medical Education: A Review of the Literature with Comments from the Law", Academic Medicine 1996 January (Supp); 71 (1): S113-S118. Norman, G.R. and Schmidt, H.G. "The Psychological Basis of Problem-Based Learning: A Review of the Evidence", Academic Medicine 1992 September; 67 (9): 557-65. Reynolds, C.F.; Adler, S.; Kanter, S.L.; Horn, J.P.; Harvey, J.; Bernier, G.M. "The Undergraduate Medical Curriculum: Centralized vs. Departmentalized", Academic Medicine 1995 August; 70 (8): 671-5. Rivo, M.L.; Mays, H.L.; Katzoff, J.; Kindig, D.A. "Managed Health Care: Implications for the Physician Workforce and Medical Education", Journal of the American Medical Association, 1995 September 6; 274 (9): 712-15. Schmidt, H.G. "Foundations of Problem-Based Learning: Some Explanatory Notes", Medical Education 1993 (27): 422-32. Sheets, K.J.; Anderson, W.A.; Alguire, P.C. "Curriculum Development and Evaluation in Medical Education", Journal of General Internal Medicine 1992 September-October; 7 (5): 538-43. Smith, E.G. "Study of Functioning and Location of US Citizen Foreign Medical Graduates Within the US Medical Care System", Final Report (Volumes I and II), Health Resources Administration; 1983: 1-226. Souba, W.W. "Professionalism, Responsibility and Service in Academic Medicine", Surgery 1996 January; 119 (1): 1-8. Sutnick, A.I.; Stillman, P.L.; Norcini, J.J.; Friedman, M.; Williams, R.G.; Trace, D.A.; Schwartz, M.A.; Wang, Y.; Wilson, M.P. "Pilot Study of the Use of the ECFMG Clinical Competence Assessment to Provide Profiles of Clinical Competencies of Graduates of Foreign Medical Schools for Residency Directors", Academic Medicine 1994 January; 69 (1): 65-67. Their, S.O. "Academic Medicines Choices in an Era of Reform", Academic Medicine 1994 March; 69 (3): 185-9. Toscani, M.R. and Patterson, R. "Evaluating and Creating Effective Patient Education Programs", Drug Benefit Trends 1995; 7 (9): 36-9, 44. Veloski, J.; Barzansky, B.; Nash, D.B.; Bastacky, S.; Stevens, D.P. "Medical Student Education in Managed Care Settings", Journal of the American Medical Association 1996 September 4; 276 (9): 667-71. Vernon, D.T. and Blake, R.L. "Does Problem-Based Learning Work? A Meta-Analysis of Evaluative Research", Academic Medicine 1993 July; 68 (7): 550-63. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||