A response to paper "systems thinking" by D. Cabrera et al.: additional thoughts on systems thinking

The paper by Cabrera, Colosi, and Lobdell presents a valuable explanation of ST and its relationship to evaluation. Given the complexity of the processes addressed by evaluators, it seems evident that ST is a valuable perspective and tool for evaluators as it facilitates the perception, analysis, and communication of complex issues and topics. It is my interpretation that the practice of evaluation may be becoming increasingly oriented to the future. I want to thank the authors for sharing their insights regarding ST and evaluation. I hope newcomers to ST find my comments useful.     

Making sense of the emerging conversation in evaluation about systems thinking and complexity science

In the last twenty years, a conversation has emerged in the evaluation field about the potential of systems thinking and complexity science (STCS) to transform the practice of evaluating social interventions. Documenting and interpreting this conversation are necessary to advance our understanding of the significance of using STCS in planning, implementing, and evaluating social interventions. Guided by a generic framework for evaluation practice, this paper reports on an inter-disciplinary literature review and argues that STCS raises some new ways of thinking about and carrying out the following six activities: 1) supporting social problem solving; 2) framing interventions and contexts; 3) selecting and using methods; 4) engaging in valuing; 5) producing and justifying knowledge; and 6) facilitating use. Following a discussion of these issues, future directions for research and practice are suggested.     

Systems thinking

Evaluation is one of many fields where “systems thinking” is popular and is said to hold great promise. However, there is disagreement about what constitutes systems thinking. Its meaning is ambiguous, and systems scholars have made diverse and divergent attempts to describe it. Alternative origins include: von Bertalanffy, Aristotle, Lao Tsu or multiple aperiodic “waves.” Some scholars describe it as synonymous with systems sciences (i.e., nonlinear dynamics, complexity, chaos). Others view it as taxonomy—a laundry list of systems approaches. Within so much noise, it is often difficult for evaluators to find the systems thinking signal. Recent work in systems thinking describes it as an emergent property of four simple conceptual patterns (rules). For an evaluator to become a “systems thinker”, he or she need not spend years learning many methods or nonlinear sciences. Instead, with some practice, one can learn to apply these four simple rules to existing evaluation knowledge with transformative results.     

On lateral thinking

Lateral thinking led to results on the measurement of risk being applied to the problem of measuring income inequality. In this, and in other areas of economics, lateral thinking has made a significant contribution. At the same time, there are dangers. The parallel may be deceptive. The paper examines two ways in which a straightforward translation of techniques from one field to another can be misleading and one example where lateral thinking about multi-dimensional inequality has considerable promise.     

A comparison of linear and systems thinking approaches for program evaluation illustrated using the Indiana Interdisciplinary GK-12

Logic models are based on linear relationships between program resources, activities, and outcomes, and have been used widely to support both program development and evaluation. While useful in describing some programs, the linear nature of the logic model makes it difficult to capture the complex relationships within larger, multifaceted programs. Causal loop diagrams based on a systems thinking approach can better capture a multidimensional, layered program model while providing a more complete understanding of the relationship between program elements, which enables evaluators to examine influences and dependencies between and within program components. Few studies describe how to conceptualize and apply systems models for educational program evaluation. The goal of this paper is to use our NSF-funded, Interdisciplinary GK-12 project: Bringing Authentic Problem Solving in STEM to Rural Middle Schools to illustrate a systems thinking approach to model a complex educational program to aid in evaluation. GK-12 pairs eight teachers with eight STEM doctoral fellows per program year to implement curricula in middle schools. We demonstrate how systems thinking provides added value by modeling the participant groups, instruments, outcomes, and other factors in ways that enhance the interpretation of quantitative and qualitative data. Limitations of the model include added complexity. Implications include better understanding of interactions and outcomes and analyses reflecting interacting or conflicting variables.     

Extending systems thinking in planning and evaluation using group concept mapping and system dynamics to tackle complex problems

Group concept mapping (GCM) has been successfully employed in program planning and evaluation for over 25 years. The broader set of systems thinking methodologies (of which GCM is one), have only recently found their way into the field. We present an overview of systems thinking emerging from a system dynamics (SD) perspective, and illustrate the potential synergy between GCM and SD. As with GCM, participatory processes are frequently employed when building SD models; however, it can be challenging to engage a large and diverse group of stakeholders in the iterative cycles of divergent thinking and consensus building required, while maintaining a broad perspective on the issue being studied. GCM provides a compelling resource for overcoming this challenge, by richly engaging a diverse set of stakeholders in broad exploration, structuring, and prioritization. SD provides an opportunity to extend GCM findings by embedding constructs in a testable hypothesis (SD model) describing how system structure and changes in constructs affect outcomes over time. SD can be used to simulate the hypothesized dynamics inherent in GCM concept maps. We illustrate the potential of the marriage of these methodologies in a case study of BECOMING, a federally-funded program aimed at strengthening the cross-sector system of care for youth with severe emotional disturbances.     

"Race" as an Interaction Order Phenomenon: W.E.B. Du Bois's "Double Consciousness" Thesis Revisited

This article reports on a study of interaction between Americans who self-identify as Black and White that reveals underlying expectations with regard to conversation that differ between the two groups. These differences seem not to have much to do with class or gender, but rather vary largely according to self-identification by "race." The argument of this paper will be that the social phenomena of "race" are constructed at the level of interaction whenever Americans self-identified as Black and White speak to one another. This is because the Interaction Order expectations with regard to both self and community vary between the two groups. Because the "language games" and conversational "preferences" practiced by the two groups are responsive to different Interaction Orders, the "working consensus" is substantially different, and as a consequence, conversational "moves" are not recognizably the same. It will be argued that a great deal of institutional discrimination against African Americans can be traced to this source.     

Irrational thinking among slot machine players

According to the cognitive perspective on gambling, regular gamblers persist in trying to win money at gambling because they hold a set of false beliefs about the nature of gambling, the likelihood of winning, and their own expertise. In order to investigate this claim, twenty seven university students were recruited who played one of three types of games at least twice a week: slot machines, video draw poker, and video amusement games. Subjects played their preferred machines on site (clubs, hotels and amusement arcades) first for at least thirty minutes and then the other two games for a minimum of twenty minutes each. During play, each subject spoke aloud into a microphone describing what he or she was doing or thinking about in the game. It was hypothesised that slot machine players would verbalise more irrational thinking than video poker or video amusement players and that slot machines would elicit more irrational thinking than video poker or video amusement machines. Most importantly, it was hypothesised that slot machine players would exhibit relatively greater amounts of irrational thinking when playing their preferred game. The data supported all three hypotheses. Out of all of the statements made by slot machine players when playing slot machines, 38% were categorised irrational. Furthermore, 80% of the strategic statements made by slot machine players while playing slot machines were categorised as irrational. These results are consistent with earlier work which showed high levels of irrational thinking in artificial gambling games. Together, the results provide support for a cognitive view of the origins of gambling problems.     

Women's Rights vs. "Fetal Rights":

Policies restricting the employment opportunities of women have a long history in the U.S. The most recent manifestation has been so-called "fetal protection" policies, which exclude women of childbearing age from jobs involving exposure to toxins considered dangerous to a developing fetus. Traditional arguments that women's biology is justification to keep them from jobs have resurfaced in a new form. In the present debate the issue is framed as one of competing rights, those of fetus versus those of the woman. An analysis of public policy on this issue from a feminist legal standpoint reveals how the law's implicit male standard hinders the attainment of equal employment opportunity for women, as they must now compete with hypothetical fetuses sa well sa with men.