Hindsight is 20/20: Reflections on the evolution of concept mapping

This paper considers the origins and development of the concept mapping methodology, a summary of its growth, and its influence in a variety of fields. From initial discussions with graduate students, through the rise of the theory-driven approach to program evaluation and the development of a theoretical framework for conceptualization methodology, the paper highlights some of the key early efforts and pilot projects that culminated in a 1989 special issue on the method in Evaluation and Program Planning that brought the method to the attention of the field of evaluation. The paper details the thinking that led to the standard version of the method (the analytic sequence, “bridging” index, and pattern matching) and the development of the software for accomplishing it. A bibliometric analysis shows that the rate of citation continues to increase, where it has grown geographically and institutionally, that the method has been used in a wide variety of disciplines and specialties, and that the literature had an influence on the field. The article concludes with a critical appraisal of some of the key aspects of the approach that warrant further development.     

Use of multiple cluster analysis methods to explore the validity of a community outcomes concept map

Concept mapping is now a commonly-used technique for articulating and evaluating programmatic outcomes. However, research regarding validity of knowledge and outcomes produced with concept mapping is sparse. The current study describes quantitative validity analyses using a concept mapping dataset. We sought to increase the validity of concept mapping evaluation results by running multiple cluster analysis methods and then using several metrics to choose from among solutions. We present four different clustering methods based on analyses using the R statistical software package: partitioning around medoids (PAM), fuzzy analysis (FANNY), agglomerative nesting (AGNES) and divisive analysis (DIANA). We then used the Dunn and Davies-Bouldin indices to assist in choosing a valid cluster solution for a concept mapping outcomes evaluation. We conclude that the validity of the outcomes map is high, based on the analyses described. Finally, we discuss areas for further concept mapping methods research.     

And then the internet happened: Thoughts on the future of concept mapping

Over 25 years ago, in the late twentieth century, concept mapping emerged as a mixed method approach to inquiry that enables a group of people to conceptualize their thinking about a specific topic. Since then, the application of concept mapping has spread widely and an easy prediction for the future is that this trend is likely to continue; a more important and greater challenge is to think about the ways in which concept mapping may and should evolve. Discussed here are thoughts about the future of concept mapping including some predictions of likely directions and suggestions for new possibilities. Thoughts on the future are grounded in concept mapping applications that have emerged and gained ground in recent years; these include exploring wicked problems in communities and integrating concept mapping with other methods of inquiry. Thoughts on the future are also grounded in the social and cultural milieu in which we find ourselves at this time. The influence of social media and internet technologies has led to the emergence peer production and crowdsourcing as approaches to co-create information, knowledge, products and services. These tactics may create fertile ground for the further spread of concept mapping. This same collaborative milieu has produced the open software movement which in turn, offers opportunities to enhancing the methodology of concept mapping.     

Quality and rigor of the concept mapping methodology: a pooled study analysis

The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed.     

Concept mapping internal validity: A case of misconceived mapping?

Since the early 1990s, the concept mapping technique developed by William M. K. Trochim has been widely used by evaluators for program development and evaluation and proven to be an invaluable tool for evaluators and program planners. The technique combines qualitative and statistical analysis and is designed to help identify and prioritize the components, dimensions, and particularities of a given reality. The aim of this paper is to propose an alternative way of conducting the statistical analysis to make the technique even more useful and the results easier to interpret. We posit that some methodological choices made at the inception stage of the technique were ill informed, producing maps of participants’ points-of-view that were not optimal representations of their reality. Such a depiction resulted from the statistical analysis process by which multidimensional scaling (MDS) is being applied on the similarity matrix, followed by a hierarchical cluster analysis (HCA) on the Euclidian distances between statements as plotted on the resulting two-dimensional MDS map. As an alternative, we suggest that HCA should be performed first and MDS second, rather than the reverse. To support this proposal, we present three levels of argument: 1) a logical argument backed up by expert opinions on this issue; 2) statistical evidence of the superiority of our proposed approach and 3) the results of a social validation experiment.     

Perspectives on the strategic uses of concept mapping to address public health challenges

We examine the adaptation of approaches used to plan and implement the steps of concept mapping to meet specialized needs and requirements in several public health projects. Seven published concept mapping projects are detailed to document how each of the phases were modified to meet the specific aims of each project. Concept mapping was found to be a useful tool to complement public health roles such as assessment, program development, and priority setting. The phases of concept mapping allow for a blending of diverse perspectives, which is critical to public health efforts. The adaptability of concept mapping permits the use of multiple modalities such as the addition of face-to-face brainstorming; use of qualitative methods, including structured interviews; and review and use of published literature and guidelines. Another positive aspect of concept mapping for public health practice is its ability to identify program elements, provide a visual map of generated ideas and their relationships to one another, and assist in identifying priorities. Our reflections on the adaptability should help inform another generation in designing concept mapping projects and related products that may benefit from unique adaptations and the rapidly expanding social media technology and platforms.     

Using concept mapping as a planning tool: Child welfare citizen review panels

Citizen Review Panels (CRPs) are groups of citizen volunteers authorized by U.S. federal law to examine state child welfare agencies. These groups inspect policies and practices related to child protection responsibilities and are tasked with making recommendations for systemic improvement. Despite the federal mandate for each state to develop a CRP and the potential of these groups to positively impact child welfare practices, there is a dearth in the literature related to CRPs. Consequently, planning and evaluation processes of these groups vary widely. This study reports on the use of concept mapping (CM) to outline a framework for planning and subsequently evaluating the CRP in one southeastern state. CM is a mixed-method research approach that uses multi-dimensional scaling and hierarchical cluster analyses to explore an area of study. Through these analyses, the method creates visual depictions of conceptual relationships between ideas. Data yielded a seven cluster concept map that CRP members (N = 36) utilized for planning processes, and subsequently for developing an internal evaluation tool. Results from this study offer a unifying framework by which CRPs, and similar groups in other areas can utilize for planning and evaluation purposes. After a review of pertinent literature on CRPs, this article explicates CM processes utilized in this study, describes results, discusses lessons learned, and outlines apposite areas for future CRP research.     

Using concept mapping to identify opportunities for HPV vaccination efforts: Perspectives from the Midwest and West Coast

Human papillomavirus (HPV) infection significantly contributes to the burden of cancer in the United States, despite the existence of a highly effective vaccine. While numerous interventions to address vaccination uptake exist, vaccination rates remain low. We conducted a concept mapping exercise to solicit perspectives on barriers and facilitators to HPV vaccination from state-level stakeholders in five states in the Midwest and West Coast of the U.S. We identified 10 clusters of barriers and facilitators based on participants’ statements. For rural areas specifically, clusters rated as most important included education and provider influence; those rated as most feasible were education and coordinated/consistent messaging. Our results suggest that a combination of important (but potentially more difficult to implement) strategies, combined with those rated as most feasible (but potentially less impactful) may be beneficial. Our findings highlight similarities across diverse states, suggesting that states can learn from each other and work together to improve HPV vaccination rates. Using concept mapping proved to be an efficient way to collect information from diverse, stakeholders in different locations, and is a methodology that could be used for program planning in areas beyond HPV vaccination.     

Use of concurrent mixed methods combining concept mapping and focus groups to adapt a health equity tool in Canada

The aim of this project was to identify and prioritize a set of conditions to be considered for incorporating a health equity tool into public health practice. Concept mapping and focus groups were implemented as complementary methods to investigate the conditions of use of a health equity tool by public health organizations in Quebec. Using a hybrid integrated research design is a richer way to address the complexity of questions emerging from intervention and planning settings. This approach provides a deeper, operational, and contextualized understanding of research results involving different professional and organizational cultures, and thereby supports the decision-making process. Concept mapping served to identify and prioritize in a limited timeframe the conditions to be considered for incorporation into a health equity tool into public health practices. Focus groups then provided a more refined understanding of the barriers, issues, and facilitating factors surrounding the tools adoption, helped distinguish among participants’ perspectives based on functional roles and organizational contexts, and clarified some apparently contradictory results from the concept map. The combined use of these two techniques brought the strengths of each approach to bear, thereby overcoming some of the respective limitations of concept mapping and focus groups. This design is appropriate for investigating targets with multiple levels of complexity.     

Effective elements of care-physical activity initiatives for adults with a low socioeconomic status: A concept mapping study with health promotion experts

In care-physical activity (care-PA) initiatives, primary care and sports are combined to stimulate PA among adults at risk of lifestyle related diseases. Preliminary results from Dutch care-PA initiatives for adults with a low socioeconomic status (SES) indicate a decrease in participants’ body weight and an improved quality of life, however, the elements that make these initiatives successful are yet to be identified. In total, 19 Dutch health promotion experts participated in our concept mapping (CM) of the effective elements of care-PA initiatives for adults with a low SES. The experts identified 111 effective elements of these initiatives, which were grouped into 11 clusters, focusing on: 1) approaching participants within the care-PA initiative, 2) barriers experienced throughout the initiative, 3) long-term implementation, 4) customizing the care-PA initiative to the target population, 5) social support, 6) structure and guidance, 7) the professionals within the care-PA initiative, 8) the accessibility of the care-PA initiative, 9) targeted behaviour and progression, 10) recruitment and administration, and 11) intersectoral collaboration. CM was useful for creating a valuable overview of these effective elements. Our results could be used to improve the development and implementation of future care-PA initiatives for adults with a low SES.     

Concept mapping as a method to enhance evidence-based public health

In this paper we explore the suitability of concept mapping as a method for integrating knowledge from science, practice, and policy. In earlier research we described and analysed five cases of concept mapping procedures in the Netherlands, serving different purposes and fields in public health. In the current paper, seven new concept mapping studies of co-produced work are added to extend this analysis.For each of these twelve studies we analysed: (1) how the method was able to integrate knowledge from practice with scientific knowledge by facilitating dialogue and collaboration between different stakeholders in the field of public health, such as academic researchers, practitioners, policy-makers and the public; (2) how the method was able to bring theory development a step further (scientific relevance); and (3) how the method was able to act as a sound basis for practical decision-making (practical relevance).Based on the answers to these research questions, all but one study was considered useful for building more evidence-based public health, even though the extent to which they underpinned actual decision-making varied. The chance of actually being implemented in practice seems strongly related to the extent to which the responsible decision-makers are involved in the way the concept map is prepared and executed.     

Made wrong,excluded, and ignored: Introduction to a special issue on lesbians and the law

Abstract

For some time now, the specific experiences that lesbians have with the law have been considerably overlooked. This is because there is a focus on how gay males have contact with the law, as gay males are the group whose same-sex intimacies are most often criminalized in legislative frameworks. It is rare for the same-sex intimacies of lesbians to be targeted explicitly in legislative frameworks. This special issue therefore focuses on how lesbians have contact with the law. It features articles focused on what the guest editors conceptualize as how lesbians are being made wrong, being excluded, and being ignored by the law, and each of these are outlined in further detail in this introduction. The articles highlight the relational nexus between lesbians and different frameworks, processes, rules, policies, institutions and applications of law, and the far-reaching impacts this nexus produces in the lives of lesbians around the world.     

Social network analysis: New ethical approaches through collective reflexivity. Introduction to the special issue of Social Networks

Research in social network analysis (SNA) faces unprecedented ethical challenges today due to both technological developments (‘big’ data) and a growing tendency toward institutionalization of ethics governance. We argue that a suitable response requires a more comprehensive approach to SNA ethics, and we identify two main paths toward this goal. First, we need to recognize the breadth of the subject, no longer limited to anonymity and consent. Second, we need to encourage the active participation and contribution of SNA scholars. We advocate a need for greater reflexivity at both individual and collective levels, whereby researchers can learn from their own and their peers’ experiences, and foster a more ethically ‘virtuous’ approach within the whole SNA community.     

The economics of gay and lesbian couples: Introduction to a special issue on gay and lesbian households

This article presents a simple conceptual framework integrating three couple-related outcomes analyzed in this volume: wage differentials in earnings related to couple formation, household formation (including cohabitation and registration as Registered Domestic Partnership), and intra-household allocation of income. It also discusses some of the articles’ main findings.

Evaluation of the impact of intervention programmes on education organisations: Application to a Quality Management System

This paper analyses the importance of evaluating the various components of the programmes or actions carried out by education organisations. It highlights the need to assess the impact of the intervention on the organisation and consider how changes are consolidated over time in interaction with the context. We propose an impact evaluation model and as an example have chosen the implementation of Quality Management Systems in schools. The paper analyses the results obtained in 40 schools in three regions (Spanish Autonomous Communities) with varying levels of implementation. The results show overall impact on these education centres as the teachers and management teams of the centres perceive it. This impact is more evident in some of the dimensions considered in the study than in others. The results also confirm the differences between regional contexts.