The interpretation and use of mixed methods research within programme evaluation practice

[摘要] The contemporary evaluation literature advocates for and recommends a pluralistic approach toprogramme evaluation, with some writers contending that the use of multiple and/or mixedmethods for the practice is inevitable. The rationale for such an approach encompassesaspects of both the 'technical' and the 'political' requirements of evaluation practice. A review ofevaluation research literature underscores the important role of mixed methods researchtowards realizing richer evaluation findings, and addressing the pragmatic, democratic andpolitical facets of the evaluation practice. However, it is observed that there is a dearth ofliterature that focuses on how the use of a mixed methods evaluation approach facilitates therealization of richer conclusions or inferences about programme merit/worth. Thus, theoverarching aim of the thesis is to establish how the perception and implementation of mixedmethods research among evaluation practitioners influences the nature of inferences theymake.This thesis aims at identifying patterns and relationships within and between conceptions andpractices of mixed methods evaluation through a descriptive process. The selection of cases istherefore purposive and includes fourteen published evaluation articles onprojects/programmes. An analytical framework is developed on the basis of a literature reviewon mixed methods research and background literature on evaluation research. This frameworkguides the qualitative content analysis of each case study and the cross-case analysis acrossthe fourteen studies to identify common patterns.The findings reveal two prominent perspectives of mixed methods evaluation prevailing amongevaluation practitioners. The first (labeled a 'strong' conception) has the intention of and placesemphasis on the integration of the qualitative and quantitative components, with the primaryobjective of obtaining richer evaluation inferences. In this conception, the use of the methodsand the data/inferences thereof are synthesized to achieve this goal. This conception iscongruent with mixed methods purposes of: - 'complementarity' and 'triangulation' and isresponsive to the 'technical' needs of evaluation. The second perspective (labeled a 'weak'conception) is silent about the integration of the respective methods or data/findings/inferences,qualifying the use of multiple methods and data in a single study as sufficing for a mixedmethods approach. It resonates with justifications of mixed methods research that addressissues of: - comprehensiveness, multiple view points, inclusiveness and democracy and seemsmore tailored to the 'political' needs of evaluation. The findings also reveal that the resultingmultiple inferences from this 'weak' conception can weaken each other when contradicting orinaccurate qualitative and quantitative findings result, especially when the complimentaryfunction of either method is not planned a priori.Therefore within the context of realizing richer and more valid evaluation findings/inferences, itis recommended that the purposes and qualification as mixed methods research of the secondperspective be re-considered. It is apparent that in embracing the 'political' needs of evaluationpractice, this conception seems to eschew the 'technical' requirements initially intended for amixed methods approach. This has implications particularly for the mixed methods purpose of'expansion' and rationales of pluralism, inclusiveness and democracy, which are seeminglypopular within programme evaluation practice.

[发布机构] Stellenbosch University