Quality Control for Scientific Research: Addressing Reproducibility,Responsiveness, and Relevance

ABSTRACT

Efforts to address a reproducibility crisis have generated several valid proposals for improving the quality of scientific research. We argue there is also need to address the separate but related issues of relevance and responsiveness. To address relevance, researchers must produce what decision makers actually need to inform investments and public policy—that is, the probability that a claim is true or the probability distribution of an effect size given the data. The term responsiveness refers to the irregularity and delay in which issues about the quality of research are brought to light. Instead of relying on the good fortune that some motivated researchers will periodically conduct efforts to reveal potential shortcomings of published research, we could establish a continuous quality-control process for scientific research itself. Quality metrics could be designed through the application of this statistical process control for the research enterprise. We argue that one quality control metric—the probability that a research hypothesis is true—is required to address at least relevance and may also be part of the solution for improving responsiveness and reproducibility. This article proposes a “straw man” solution which could be the basis of implementing these improvements. As part of this solution, we propose one way to “bootstrap” priors. The processes required for improving reproducibility and relevance can also be part of a comprehensive statistical quality control for science itself by making continuously monitored metrics about the scientific performance of a field of research.     

U.K. Foot and Mouth Disease: A Systemic Risk Assessment of Existing Controls

This article details a systemic analysis of the controls in place and possible interventions available to further reduce the risk of a foot and mouth disease (FMD) outbreak in the United Kingdom. Using a research‐based network analysis tool, we identify vulnerabilities within the multibarrier control system and their corresponding critical control points (CCPs). CCPs represent opportunities for active intervention that produce the greatest improvement to United Kingdom's resilience to future FMD outbreaks. Using an adapted ‘features, events, and processes’ (FEPs) methodology and network analysis, our results suggest that movements of animals and goods associated with legal activities significantly influence the system's behavior due to their higher frequency and ability to combine and create scenarios of exposure similar in origin to the U.K. FMD outbreaks of 1967/8 and 2001. The systemic risk assessment highlights areas outside of disease control that are relevant to disease spread. Further, it proves to be a powerful tool for demonstrating the need for implementing disease controls that have not previously been part of the system.     

Fitting E(max) models to clinical trial dose-response data

We consider fitting Emax models to the primary endpoint for a parallel group dose–response clinical trial. Such models can be difficult to fit using Maximum Likelihood if the data give little information about the maximum possible response. Consequently, we consider alternative models that can be derived as limiting cases, which can usually be fitted. Furthermore we propose two model selection procedures for choosing between the different models. These model selection procedures are compared with two model selection procedures which have previously been used. In a simulation study we find that the model selection procedure that performs best depends on the underlying true situation. One of the new model selection procedures gives what may be regarded as the most robust of the procedures.     

Atheism,Secularity, and Well‐Being: How the Findings of Social Science Counter Negative Stereotypes and Assumptions

What do we currently know about atheists and secular people? In what ways are atheism and secularity correlated with positive societal outcomes? This article offers a thorough presentation and discussion of the latest social scientific research concerning the identities, values, and behaviors of people who don’t believe in God or are non‐religious, and addresses the ways in which atheism and secularity are positively correlated with societal well‐being.     

Rural Community and Rural Resilience: What is important to farmers in keeping their country towns alive?

Many studies have highlighted the phenomenon of rural decline in parts of the developed world, summarised as a loss in agricultural employment leading to a decline in the number and size of rural settlements. This study of small towns in part of Australia’s inland rural “heartland” employs the concepts of interactional rural community of place and rural resilience to identify how farmers perceive their interactions with their local town. This study suggests that robust levels of ongoing engagement between farmers and town communities are important in maintaining rural populations and services along with both a strong local economy and environment. Face-to-face interviews with 115 farmers in two rural regions of New South Wales, Australia, highlight the importance of the local economy and jobs, the quality of the local environment and a strong sense of belonging, in contributing to a strong sense of local community and potential for resilience.     

Study abroad and cultural immersion: an alumni retrospect

ABSTRACT

This paper draws on an analysis of 2600 study abroad alumni responding to a survey that spans more than 25 years. Our inquiry focuses on evaluating how programme length, housing arrangements, number, and type of co-curricular activities impact alumni’s attributions of personal growth and their attitudes towards global engagement and future social relationships. This paper is also informed by a 5-year alumni follow-up study. Our study finds that the type of housing and participation in co-curricular activities have a significant moderating effect on alumni attributions. With rare exception, programme length does not have a significant impact on reported outcome measures. We conclude the paper with a discussion of how these findings can help guide future research and administrative programming.     

Field Placement and the Impact of Financial Stress on Social Work and Human Service Students

While significant research has been undertaken exploring the pedagogical benefits of undertaking lengthy social work and human services field placements, there has been very little consideration given to the potential financial stress involved for students. This study sought to address this knowledge gap. Research was conducted in 2014 using quantitative and qualitative methods with students and academic and professional staff from six Queensland universities. The findings show a significant relationship between unpaid placements and financial hardship, creating considerable stress for students and at times a compromised placement learning experience. The limited flexibility in requirements from professional bodies and universities for how placements are undertaken has been identified as a key contributor to financial hardship. Addressing the complexities inherent in this issue requires a collaborative effort from multiple stakeholders and should not be regarded as a problem for students to endure and manage.     

Religiosity as a Potential Moderator of the Effects of R-rated Movies on Delinquency

Previous research suggests viewing R-rated movies is associated with delinquency (Tanski et al. 2010 Tanski , Susanne E. , Sonya Dal Cin , Mike Stoolmiller , and James D. Sargent . 2010. “Parental R-Rated Movie Restriction and Early-Onset Alcohol Use.” Journal of Studies on Alcohol and Drugs 71(3):452–460.[PubMed], [Web of Science ®] , [Google Scholar]; Titus-Ernstoff et al. 2008 Titus-Ernstoff , L. , M. A. Dalton , A. M. Adachi-Mejia , M. R. Longacre , and M. L. Beach . 2008 . “Longitudinal Study of Viewing Smoking in Movies and Initiation of Smoking by Children.” Pediatrics 121 : 15 – 21 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]), while religiosity leads to decreases in substance use (Jang and Johnson 2001 Jang , Sung Joon and Byron R. Johnson . 2001 . “Neighborhood Disorder, Individual Religiosity, and Adolescent Use of Illicit Drugs: A Test of Multilevel Hypotheses.” Criminology 39 ( 1 ): 109 – 144 .[Crossref], [Web of Science ®] , [Google Scholar]; Kovacs, Piko, and Fitzpatrick 2011 Jang , Sung Joon and Byron R. Johnson . 2011 . “The Effects of Childhood Exposure to Drug Users and Religion on Drug Use in Adolescence and Young Adulthood.” Youth and Society 43 : 1220 – 1245 .[Crossref], [Web of Science ®] , [Google Scholar]; Miller 1998 Miller , W. 1998 . “Researching the Spiritual Dimensions of Alcohol and Other Drug Problems.” Addiction 93 : 979 – 990 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]; Regnerus 2003 Regnerus , Mark D. 2003 . “Moral Communities and Adolescent Delinquency: Religious Contexts and Community Social Control.” Sociological Quarterly 44 : 523 – 554 .[Crossref], [Web of Science ®] , [Google Scholar]). However, the influences of religiosity and viewing R-rated movies have not been examined in conjunction. This article tests whether religiosity moderates the effects of R-rated movies on delinquency, results suggesting that rather than religiosity moderating the effects of R-rated movies on delinquency, viewing R-rated movies actually tempers the pro-social effects of religiosity. This finding only held true for certain types of substance abuse, and not activity-based forms of delinquency such as fighting. The results are discussed in light of their implications for the current study of the effects of religiosity on delinquency, and suggestions are made for future research on the relationship between viewing R-rated movies and delinquency.