A comparison of linear and systems thinking approaches for program evaluation illustrated using the Indiana Interdisciplinary GK-12

Logic models are based on linear relationships between program resources, activities, and outcomes, and have been used widely to support both program development and evaluation. While useful in describing some programs, the linear nature of the logic model makes it difficult to capture the complex relationships within larger, multifaceted programs. Causal loop diagrams based on a systems thinking approach can better capture a multidimensional, layered program model while providing a more complete understanding of the relationship between program elements, which enables evaluators to examine influences and dependencies between and within program components. Few studies describe how to conceptualize and apply systems models for educational program evaluation. The goal of this paper is to use our NSF-funded, Interdisciplinary GK-12 project: Bringing Authentic Problem Solving in STEM to Rural Middle Schools to illustrate a systems thinking approach to model a complex educational program to aid in evaluation. GK-12 pairs eight teachers with eight STEM doctoral fellows per program year to implement curricula in middle schools. We demonstrate how systems thinking provides added value by modeling the participant groups, instruments, outcomes, and other factors in ways that enhance the interpretation of quantitative and qualitative data. Limitations of the model include added complexity. Implications include better understanding of interactions and outcomes and analyses reflecting interacting or conflicting variables.     

Probabilistic Risk Analysis and Terrorism Risk

Since the terrorist attacks of September 11, 2001, and the subsequent establishment of the U.S. Department of Homeland Security (DHS), considerable efforts have been made to estimate the risks of terrorism and the cost effectiveness of security policies to reduce these risks. DHS, industry, and the academic risk analysis communities have all invested heavily in the development of tools and approaches that can assist decisionmakers in effectively allocating limited resources across the vast array of potential investments that could mitigate risks from terrorism and other threats to the homeland. Decisionmakers demand models, analyses, and decision support that are useful for this task and based on the state of the art. Since terrorism risk analysis is new, no single method is likely to meet this challenge. In this article we explore a number of existing and potential approaches for terrorism risk analysis, focusing particularly on recent discussions regarding the applicability of probabilistic and decision analytic approaches to bioterrorism risks and the Bioterrorism Risk Assessment methodology used by the DHS and criticized by the National Academies and others.     

A meta-analytic examination of the potential correlates and consequences of workload

Over the last four decades, occupational stress researchers have given considerable attention to the potential correlates and consequences of workload. In the current study, we use meta-analysis (overall k = 336) to quantitatively review the workload literature. In analyses of hypothesized correlates, we found that social support was negatively associated (ρ = ?.20 for supervisor support; ρ = –.11 for co-worker support) and that trait negative affectivity (ρ = .22), role ambiguity (ρ = .28), role conflict (ρ = .44) and work-family conflict (ρ = .44 for work-to-family conflict; ρ = .20 for family-to-work conflict) were each positively associated with workload. Analyses examining hypothesized outcome variables suggest that workload is negatively associated with several indices of psychological and physical well-being (ρs were generally in the –.20s and –.30s), and affective organizational commitment (ρ = –.11), and is positively associated with turnover intention (ρ = .16) and absenteeism (ρ = .07).     

A framework for providing telecommuting as a reasonable accommodation: some considerations on a comparative case study

Telecommuting, whether full time, part time, or over short periods when the need arises, can be an important accommodation for employees with disabilities. Indeed, telecommuting may be the only form of accommodation that offers employees whose disabilities fluctuate a means to stay consistently and gainfully employed. This article describes one employer's experience in considering a request for telecommuting as a reasonable accommodation for a particular employee. Drawing on real-life examples, both positive and negative, this article provides a win/win framework for decision-making that can help employers evaluate the use of telecommuting as a possible accommodation and facilitates open and ongoing communication between employer and employee.     

Sad to say: is it time for Sorrow Management Theory?

Terror Management Theory (TMT) research often asks respondents to describe their personal death. This exposure enables the testing of hypotheses regarding defenses ("buffers") against the anxious awareness of death. Curiously, though, the respondents' narratives are not analyzed or reported. The present study offers a qualitative analysis of 209 college student narrative responses provided for this purpose by TMT researchers Jeff Greenberg and Nathan Heflick. The narratives are reviewed with attention to affect (adjectives), semantic structure, and theme. Many responses are marked by anxiety, but sorrow is also pervasive. Individual differences in response substance and structure invite further exploration. It is suggested that our understanding of the response to personal death threat could be enhanced by encompassing sorrow as well as anxiety within the same conceptual framework. Several specific hypotheses and suggestions are offered.     

Response

This article focuses on the roles consulting units can play in statistics departments, universities, and the statistics profession. An opinion on what constitutes legitimate academic activity is given. The roles endorsed are consistent with the view that statistics is both a science and an integral part of scientific methodology and hence that statistics departments need to be concerned with both the development of new methods and the proper application of those methods in the pure and applied sciences. Consulting units serve as an interface between the developers and users of statistical methodology. Communication between developers and users can be carried on effectively through consultants who are actively involved in the education of users and in the education of statistics students, either formally or informally. Such communication is also enhanced when consultants are publishing in both statistics and nonstatistics journals. Routine data analysis is not endorsed as a justifiable academic activity for consulting faculty. Supervision of graduate students on such analyses, however, can be justified. The success a consulting unit can have in playing its roles depends on appropriate funding, staffing, and administration of the unit.     

Paperwork and bureaucracy

This study explores paperwork as an inherent characteristic of bureaucratic behavior. The magnitude and scope of the federal government paperwork burden on the private sector is given particular emphasis. A theoretical model of bureaucratic behavior is developed which shows that bureaucrats employ paperwork to shift the cost of agency functions to the private sector in order to increase their perquisites of office. The model indicates that, if the private sector were compensated for the federal paperwork burden, agency employment would be smaller, as would agency output and the volume of private-sector labor expended in preparing federal forms.     

Intake fraction for multimedia pollutants: a tool for life cycle analysis and comparative risk assessment.

We employ the intake fraction (iF) as an effective tool for expressing the source-to-intake relationship for pollutant emissions in life cycle analysis (LCA) or comparative risk assessment. Intake fraction is the fraction of chemical mass emitted into the environment that eventually passes into a member of the population through inhalation, ingestion, or dermal exposure. To date, this concept has been primarily applied to pollutants whose primary route of exposure is inhalation. Here we extend the use of iF to multimedia pollutants with multiple exposure pathways. We use a level III multimedia model to calculate iF for TCDD and compare the result to one calculated from measured levels of dioxin toxic equivalents in the environment. We calculate iF for emissions to air and surface water for 308 chemicals. We correlate the primary exposure route with the magnitudes of the octanol-water partition coefficient, Kow, and of the air-water partitioning coefficient (dimensionless Henry constant), Kaw. This results in value ranges of Kow and Kaw where the chemical exposure route can be classified with limited input data requirements as primarily inhalation, primarily ingestion, or multipathway. For the inhalation and ingestion dominant pollutants, we also define empirical relationships based on chemical properties for quantifying the intake fraction. The empirical relationships facilitate rapid evaluation of many chemicals in terms of the intake. By defining a theoretical upper limit for iF in a multimedia environment we find that iF calculations provide insight into the multimedia model algorithms and help identify unusual patterns of exposure and questionable exposure model results.     

Finding Known Journal Articles: Comparing a Citation Linker to an A-to-Z List

OpenURL links provide access to full-text articles from citation databases; however, end users who have found citations outside of library databases must find the full text of their journal articles another way. Many OpenURL link resolvers offer a citation finder service which allows the end user to search for specific known items at the article level. This paper studies the usability of the “citation linker” search from Ex Libris' SFX®. Twenty-one volunteers tested finding known journal articles using both a standard A-to-Z list and SFX' citation linker search. The researchers compare results from the two search interfaces, document the problems found, recommend which interface makes the best default search interface, and suggest improvements to the citation linker interface.