Three Recommendations for Improving the Use of p-Values

ABSTRACT

Researchers commonly use p-values to answer the question: How strongly does the evidence favor the alternative hypothesis relative to the null hypothesis? p-Values themselves do not directly answer this question and are often misinterpreted in ways that lead to overstating the evidence against the null hypothesis. Even in the “post p?<?0.05 era,” however, it is quite possible that p-values will continue to be widely reported and used to assess the strength of evidence (if for no other reason than the widespread availability and use of statistical software that routinely produces p-values and thereby implicitly advocates for their use). If so, the potential for misinterpretation will persist. In this article, we recommend three practices that would help researchers more accurately interpret p-values. Each of the three recommended practices involves interpreting p-values in light of their corresponding “Bayes factor bound,” which is the largest odds in favor of the alternative hypothesis relative to the null hypothesis that is consistent with the observed data. The Bayes factor bound generally indicates that a given p-value provides weaker evidence against the null hypothesis than typically assumed. We therefore believe that our recommendations can guard against some of the most harmful p-value misinterpretations. In research communities that are deeply attached to reliance on “p?<?0.05,” our recommendations will serve as initial steps away from this attachment. We emphasize that our recommendations are intended merely as initial, temporary steps and that many further steps will need to be taken to reach the ultimate destination: a holistic interpretation of statistical evidence that fully conforms to the principles laid out in the ASA statement on statistical significance and p-values.     

Bioecological Implications of Narrative Exposure Therapy in Low‐Resource Settings: Individual,Family, Community,and Socio‐Political Contexts

Traumatic stress can have detrimental effects on individuals, families, and communities. Narrative exposure therapy (NET) is an evidence‐based intervention for decreasing individuals’ post‐traumatic stress disorder (PTSD) symptoms and has been tested in some of the most challenging contexts, such as in post‐conflict refugee camps. Although the focus of NET is on reducing individual PTSD symptoms, the impact of NET can be seen beyond the individual level. The purpose of this paper is to examine some of the ecological implications of using NET with trauma‐affected populations in low‐resource settings. We describe the implications of NET at the family, community, and sociopolitical levels using several case examples. Finally, we outline limitations and future directions for improving the delivery of NET in settings with limited resources.     

On Hotelling’s T2 test in a special paired sample case

In a special paired sample case, Hotelling’s T² test based on the differences of the paired random vectors is the likelihood ratio test for testing the hypothesis that the paired random vectors have the same mean; with respect to a special group of affine linear transformations it is the uniformly most powerful invariant test for the general alternative of a difference in mean. We present an elementary straightforward proof of this result. The likelihood ratio test for testing the hypothesis that the covariance structure is of the assumed special form is derived and discussed. Applications to real data are given.     

Linking Migration Intentions with Flows: Evidence and Potential Use

In the absence of reliable, internationally available migration flow data necessary for statistical forecasting, policymakers increasingly turn to survey data on emigration intentions to evaluate future migration trends. The important assumption – i.e. that there is a measurable and systematic relationship between the intention to migrate and actual migration – has not been firmly established at the international level. We examine the association between estimated population averages of emigration intentions and official migration flow data based on data for more than 160 countries. The results show a strong association between emigration intentions and recorded bilateral flows to industrialized countries, as well as between intentions and aggregated out‐migration. The results provide policymakers with a reliability assessment of survey data on emigration intentions and encourage future attempts to incorporate survey data in formal statistical migration forecasting models.     

WILL GIRLS BE GIRLS? RISK TAKING AND COMPETITION IN AN ALL‐GIRLS' SCHOOL

We conduct an experiment that examines the relationship between girls only schooling and risk taking and competitive behavior. In it, we compare decisions made by students in an all‐girls' school to those made by students in a closely matched coeducational school. We further investigate the developmental nature of this behavior by comparing choices made by younger students (Grades 7 and 8) with those of older students (Grades 11 and 12). We focus on the differences between those who select into the all‐girls' school, and find that although girls educated in a single sex environment are the most risk averse, they are also among the most competitive. These results lend texture to the hypothesis that “nurture matters” in the gender differences debate. (JEL C93, J16, I21)     

Quality of Life Enhancement Research Program: Lessons Learned

The attempted Quality of Life Enhancement program appeared to be a novel method for creating a symbiotic and sustainable relationship between a large Southeastern university and local nursing home and assisted living facilities (ALF), through which ALF residents would attend university arts and sporting events. During implementation, it was discovered that this project was unsustainable, undesired, unneeded by many care centers, difficult to implement, and required a dedicated and specialized staff. After attempting to enroll 14 local care facilities, only 2 were interested in becoming involved and produced 5 eligible participants. During the program itself, coordination with facilities and residents was difficult to maintain and unique transportation and accommodation needs were challenging to overcome. Finally, data collection was time consuming and generally unfruitful. Ultimately the project was discontinued after a year of attempted recruitment and implementation of inclusive changes to protocol. While creating an ongoing relationship with local care facilities and local universities may enhance opportunities for residents and research into important areas, such relationships take time, effort, and specialized staff to maintain.     

CEO burnout,managerial discretion,and firm performance: The role of CEO locus of control,structural power,and organizational factors

Despite the possibility of burnout resulting from dynamics in firms' upper echelons, little if any work has focused on chief executive officer's (CEO's) burnout and firm performance. Drawing on managerial discretion theory, this article analyzes the influence of CEO burnout on firm performance and the moderating roles of the individual (CEO locus of control), structural power (CEO duality and CEO tenure), and organizational characteristics (size, age, and resource availability) related to managerial discretion. Using a sample of 156 CEOs in Swedish firms, we find a negative association between CEOs who report higher burnout and firm performance. Our results confirm that CEO duality and resource availability ameliorate and firm size exacerbates the negative association between CEO burnout and firm performance. Contrary to our expectations, CEO locus of control, CEO tenure, and firm age do not influence this relationship. We discuss the implications of our research for upper echelons theory and strategic leadership theory.     

Machine Learning for Drug Overdose Surveillance

ABSTRACT

We describe two recently proposed machine learning approaches for discovering emerging trends in fatal accidental drug overdoses. The Gaussian Process Subset Scan (Herlands, McFowland, Wilson, & Neill, 2017 Neill, D. B. (2017). Multidimensional tensor scan for drug overdose surveillance. Journal of Public Health Informatics, 9(1), e20. doi:10.5210/ojphi.v9i1.7598[Crossref] , [Google Scholar]) enables early detection of emerging patterns in spatio-temporal data, accounting for both the complex, correlated nature of the data and the fact that detecting subtle patterns requires integration of information across multiple spatial areas and multiple time steps. We apply this approach to 17 years of county-aggregated data for monthly opioid overdose deaths in the New York City metropolitan area, showing clear advantages in the utility of discovered patterns as compared to typical anomaly detection approaches. To detect and characterize emerging overdose patterns that differentially affect a subpopulation of the data, including geographic, demographic, and behavioral patterns (e.g., which combinations of drugs are involved), we apply the Multidimensional Tensor Scan (Neill, 2017 Neill, D. B. (2017). Multidimensional tensor scan for drug overdose surveillance. Journal of Public Health Informatics, 9(1), e20. doi:10.5210/ojphi.v9i1.7598[Crossref] , [Google Scholar]) to 8 years of case-level overdose data from Allegheny County, Pennsylvania. We discover previously unidentified overdose patterns which reveal unusual demographic clusters, show impacts of drug legislation, and demonstrate potential for early detection and targeted intervention. These approaches to early detection of overdose patterns can inform prevention and response efforts, as well as understanding the effects of policy changes.     

Tracking of sport and exercise types from midlife to old age: a 20-year cohort study of British men

Background

Previous physical activity (PA) tracking studies have examined the stability of overall PA and/or PA types, but few have investigated how specific types of sport/exercise track over the life course. The aim of this study was to determine how specific sports/exercises in midlife track and predict future sport/exercise and PA in men transitioning to old age.

Methods

Seven thousand seven hundred thirty-five men (aged 40–59?years) recruited in 1978–80 were followed up after 12, 16 and 20?years. At each wave men self-reported participation in sport/exercise. Frequent sport/exercise participants (>?1/month) reported the types of sport/exercise they engaged in. Men also reported total PA, health status, lifestyle behaviours and socio-demographic characteristics. Stability of each sport/exercise was assessed using kappa statistics and intraclass correlation coefficients. Logistic regression estimated the odds of participating in sport/exercise and being active at 20-year follow up according to specific types of sport/exercise in midlife.

Results

Three thousand three hundred eighty-four men with complete data at all waves were included in analyses. Tracking of specific sports/exercises ranged from fair to substantial, with golf being the most common and most stable. Bowls was the most frequently adopted. Odds of participating in sport/exercise and being active in old age varied according to sport/exercise types in midlife. Golf and bowls in midlife were the strongest predictors of sport/exercise participation in old age. Golf, cricket and running/jogging in midlife were among the strongest predictors of being active in old age. Compared to participating in just one sport/exercise in midlife, sampling multiple sports/exercises was more strongly associated with sport/exercise participation and being active in old age.

Conclusion

The stability of sport/exercise participation from midlife to old age varies by type. Specific sports/exercises in midlife may be more likely to predict future PA than others. However, participating in a range of sports/exercises may be optimal for preserving PA into old age.

     

A Cost–Benefit Analysis of a Family Systems Intervention for Managing Pediatric Chronic Illness

Despite recent increases of psychosocial programs for pediatric chronic illness, few studies have explored their economic benefits. This study investigated the costs–benefits of a family systems‐based, psychosocial intervention for pediatric chronic illness (MEND: Mastering Each New Direction). A quasi‐prospective study compared the 12‐month pre–post direct and indirect costs of 20 families. The total cost for program was estimated to $5,320. Families incurred $15,249 less in direct and $15,627 less in indirect costs after MEND. On average, medical expenses reduced by 86% in direct and indirect costs, for a cost–benefit ratio of 0.17. Therefore, for every dollar spent on the program, families and their third payers saved approximately $5.74. Implications for healthcare policy and reimbursements are discussed.