RESUMEN

We conducted a systematic review of conference papers in social psychology at two large psychology conferences in Japan: the Japanese Psychological Association and the Japanese Society for Social Psychology. The conference papers were effectively not subjected to peer review; hence, they were suitable for testing if psychologists selectively reported statistically significant findings without pressure from journal editors and reviewers. We investigated the distributions of z-values converted from the p-values reported in the articles presented at the 2013 and 2018 conferences. The z-curve analyses suggest the existence of selective reporting by the authors in 2013. The expected discovery rate (EDR) was much lower than the observed discovery rate (ODR; 7% vs. 76%, respectively), and the 95% confidence interval (CI) did not include the ODR. However, this does not mean that the set of studies completely lacked evidential value. The expected replication rate (ERR) was 31%; this is significantly higher than 5%, which was expected under the null hypothesis of no effect. Changes were observed between 2013 and 2018. The ERR increased (31% to 44%), and the EDR almost doubled (7% to 13%). However, the estimation of the maximum false discovery rate (FDR; 68% in 2013 and 35% in 2018) suggested that a substantial proportion of the reported findings were false positives. Overall, while social psychologists in Japan engaged in selective reporting, this does not mean that the entire field was covered with false positives. In addition, slight signs of improvement were observed in how they reported their findings. Still, the evidential value of the target studies was weak, even in 2018, allowing for no optimism.

Asunto(s)

RESUMEN

In contemporary policy discourses, data are presented as key assets for improving health-care quality: policymakers want health care to become 'data driven'. In this article, we focus on a particular example of this ambition, namely a new Danish national quality development program for general practitioners (GPs) where doctors are placed in so-called 'clusters'. In these clusters, GPs are obliged to assess their own and colleagues' clinical quality with data derived from their own clinics-using comparisons, averages and benchmarks. Based on semi-structured interviews with Danish GPs and drawing on Science and Technology Studies, we explore how GPs understand these data, and what makes them trust-or question-a data analysis. The GPs describe how they change clinical practices based on these discussions of data. So, when and how do data for quality assurance come to influence their perceptions of quality? By exploring these issues, we carve out a role for a sociological engagement with evidence in everyday medical practices. In conclusion, we suggest a need to move from the aim of being data driven to one of being data informed.

Asunto(s)

RESUMEN

In the past three-and-a-half decades, nearly 500 randomized controlled trials (RCTs) have examined Acceptance and Commitment Therapy (ACT) for a range of health problems, including depression. However, emerging concerns regarding the replicability of scientific findings across psychology and mental health treatment outcome research highlight a need to re-examine the strength of evidence for treatment efficacy. Therefore, we conducted a metascientific review of the evidential value of ACT in treating depression. Whereas reporting accuracy was generally high across all trials, we found important differences in evidential value metrics corresponding to the types of control conditions used. RCTs of ACT compared to weaker controls (e.g., no treatment, waitlist) were well-powered, with sample sizes appropriate for detecting plausible effect sizes. They typically yielded stronger Bayesian evidence for (and larger posterior estimates of) ACT efficacy, though there was some evidence of significance inflation among these effects. RCTs of ACT against stronger controls (e.g., other psychotherapies), meanwhile, were poorly powered, designed to detect implausibly large effect sizes, and yielded ambiguous-if not contradicting-Bayesian evidence and estimates of efficacy. Although our review supports a view of ACT as efficacious for treating depression compared to weaker controls, future RCTs must provide more transparent reporting with larger groups of participants to properly assess the difference between ACT and competitor treatments such as behavioral activation and other forms of cognitive behavioral therapy. Clinicians and health organizations should reassess the use of ACT for depression if costs and resources are higher than for other efficacious treatments. Clinical trials contributing effects to our synthesis can be found at https://osf.io/qky35.

Asunto(s)

RESUMEN

A population study was carried out by collecting textile fibres present on the seats in a church, a cinema and a conference centre in Linköping, Sweden. The collection was carried out in such a way as to avoid inadvertent fibre collectives and to enable comparison of the frequency data between venues. In total 4220 fibres were examined and details of their characteristics were recorded and entered into a searchable database. Only coloured fibres over 0.5 mm in length were included in the study. 70% of the fibres were classified as cotton, 18% were man-made, 8% wool, 3% other plant and 2% other animal. Polyester and regenerated cellulose were the most abundant man-made fibres. The most frequently occurring combinations were blue and grey/black cotton, making up approximately 50% of all fibres. All other combinations were made up of less than 8% of the fibres, with red cotton being the next most prevalent. The results regarding the most frequently occurring fibre types, colours and colour/fibre type combinations are comparable to those of other population studies that have been carried out in different countries over the last 20-30 years. Additional observations regarding the frequency occurrence of certain characteristics are presented, such as differences in thickness, cross-sectional shape and presence of pigment or delustrant for the man-made fibre types.

RESUMEN

OBJECTIVE: A recent meta-review of attentional bias research in eating disorders suggests that meta-analyses and systematic reviews include many low-quality and underpowered studies (Stott et al., 2021). As such, we examined whether published research examining the link between attentional bias, using the emotional Stroop task, and eating disorders among women with eating disorders has evidential value (ruling out selective reporting of a statistically significant effect) using a p-curve analysis. A p-curve analysis plots statisticall significant p-values onto a curve from .01 to .05 to examine its distribution. We hypothesized that the p-curve would be flat, indicating no true effect. METHOD: The hypothesis, database search strategy, and data analytic approach were pre-registered. The inclusion criteria were reports that compared control and eating disorder groups, reported inferential statistics, and that used body shape/weight or general threat target words. RESULTS: Fifty published reports were included in the p-curve analyses. Unexpectedly, the half and full p-curves were significantly right-skewed, indicating evidential value. However, the results were not robust to the exclusion of the seven lowest p-values and on average, reports were underpowered. There were also 18 reports with null results (they had a p-value greater than .05), which precluded their inclusion in the p-curve analyses. DISCUSSION: The findings suggest that most of the evidence from research examining attentional biases using the emotional Stroop task among women with an eating disorder or with elevated eating disorder symptoms is underpowered and so should be interpreted with considerable caution. PUBLIC SIGNIFICANCE: Concerns have been raised about the low quality of research examining attentional biases among women with eating disorders using the emotional Stroop task. In the current research, we observed that the evidential value of primary research reporting differences between women with and without eating disorders was equivocal and had low statistical power. These results can guide researchers towards conducting more rigorous research on attentional biases among people with eating disorders.

OBJETIVO: Una meta-revisión reciente de la investigación del sesgo atencional en los trastornos de la conducta alimentaria sugiere que los metaanálisis y las revisiones sistemáticas incluyen muchos estudios de baja calidad y con poco poder estadístico (Stott et al., 2021). Como tal, examinamos si la investigación publicada que examina el vínculo entre el sesgo atencional, el uso de la tarea de Stroop emocional y los trastornos de la conducta alimentaria entre mujeres con trastornos alimentarios tiene valor de evidencia (descartando el informe selectivo de un efecto estadísticamente significativo) mediante un análisis de curva-p. Un análisis de curva-p traza p valores significativos en una curva de .01 a .05 para examinar su distribución. Planteamos la hipótesis de que la curva-p sería plana, lo que indica que no hay un efecto verdadero. MÉTODO: La hipótesis, la estrategia de búsqueda en la base de datos y el enfoque analítico de datos fueron pre-registrados. Los criterios de inclusión fueron informes que compararon grupos control y de trastornos de la conducta alimentaria, informaron estadísticas inferenciales y que utilizaron palabras clave de figura/peso corporal o de amenaza general. RESULTADOS: Cincuenta estudios publicados fueron incluidos en los análisis de la curva-p. Inesperadamente, las curvas-p medias y completas estaban significativamente sesgadas a la derecha, lo que indica un valor de evidencia. Sin embargo, los resultados no fueron robustos para la exclusión de los siete p valores más bajos y, en promedio, los reportes tuvieron poca potencia. También hubo 18 reportes con resultados nulos (tenían un p valor mayor que .05), lo que impidió su inclusión en los análisis de la curva-p. DISCUSIÓN: Los hallazgos sugieren que la mayor parte de la evidencia de la investigación que examina los sesgos atencionales utilizando la tarea de Stroop emocional entre las mujeres con un trastorno de la conducta alimentaria o con síntomas elevados del trastorno alimentario tiene poco poder y, por lo tanto, debe interpretarse con mucha precaución.

Asunto(s)

RESUMEN

In relation to the admissibility of evidence obtained using projective personality tests arose in F v. Bevándorlási és Állampolgársági Hivatam (2018). The Court of Justice of the European Union has held that an expert's report can only be accepted if it is based on the international scientific community's standards, but has refrained from stipulating what these standards are. It appears timely for European psychologists to decide what standards should be applied to determine whether or not a test is appropriate for psycholegal use. We propose standards and then apply them to the Rorschach because it was used in this case and is an exemplar of projective tests. We conclude that the Rorschach does not meet the proposed standards and that psychologists should abstain from using it in legal proceedings even in the absence of a clear judicial prohibition.

RESUMEN

A fibre population study was conducted on the parapets (a low wall along the edge of the walkway) of ten housing estates in Singapore. For each location, the ninth and tenth storeys were investigated and a total of 1256 fibres were classified according to colour and generic fibre class. Being a cosmopolitan city with tropical rainforest climate and no true distinct season, the predominant clothing style in Singapore is one that is light, comfortable and modern. Approximately half of the fibres recovered were cotton (48.3%), with polyester fibres making up the next one-third (31.4%) and rayon fibres in the third place (16.2%). Grey/black (27.7%) and blue (24.2%) constitute the top two most popular colours, together accounting for approximately half of all classified colours. The possibility of establishing subgroups of indistinguishable fibres was investigated by microscopical and fluorescence properties. The number of groups found per location varies from 1 to 3, with each group containing between 2 and 3 fibres. The findings of this study would provide the forensic fibre examiner an overall outlook on what are the abundant (or uncommon) fibre types in Singapore context, thereby assisting the examiner to assess the strength of fibre evidence in casework.

Asunto(s)

RESUMEN

The evidential value of a unique DNA database match has been extensively discussed. In principle the matter has been mathematically resolved, since the posterior odds on the match being with the trace donor are unambiguously defined. There are multiple ways to express these odds as a product of likelihood ratio and prior odds, and so the mathematics do not immediately tell us what to do in concrete cases, in particular which likelihood ratio to choose for reporting. With p the random match probability for the matching person, if innocent, and n the database size, both 1/p, originating from a suspect-centered framework, and 1/(np), originating from a database-centered framework, arise as likelihood ratio. Both have been defended and both have been criticized in the literature. We will clarify the situation by not introducing models and choices of prior probabilities until they are needed. This allows to derive the posterior odds in their most general form, which applies whenever we know that a single person among a list is not excluded as potential trace donor. We show that we need only three probabilities, that pertain to the observed match, to the database, and to the matching person respectively. How these required probabilities behave in a given context, then, differs from one situation to another. This is understandable since database searches may be done under various circumstances. They may be carried out with or without a suspect already in mind and, depending on the operational procedures, one may or may not be informed about the personal details of the person who gives the match. We show how to evaluate the required probabilities in all such cases. We will motivate why we believe that for some database searches, the 1/p likelihood ratio is more natural, whereas for others, 1/(np) seems the more sensible choice. This is not motivated by the mathematics: mathematically, the approaches are equivalent. It is motivated by considering which model best reflects the actual situation, taking into account what question was asked to begin with, and by the practical consideration of judging which likelihood ratio comes closer to the posterior odds based on the information available in the case. This article is intended to be both a research and a review article, and we end with an in-depth discussion of various arguments that have been brought forward in favor or against either 1/p or 1/(np).

Asunto(s)

RESUMEN

Currently, forensic drug experts are facing chemical identification challenges with the increasing number of new isomeric forms of psychoactive substances occurring in case samples. Very similar mass spectra for these substances could easily result in misidentification using the regular GC-MS screening methods in combination with colorimetric testing in forensic laboratories. Building on recent work from other groups, this study demonstrates that GC-VUV is a powerful technique for drug isomer differentiation, showing reproducible and discriminating spectra for aromatic ring-isomers. MS and VUV show complementary selectivity as VUV spectra are ring-position specific whereas MS spectra are characteristic for the amine moieties of the molecule. VUV spectra are very reproducible showing less than 0.1‰ deviation in library match scores and therefore small spectral differences suffice to confidently distinguish isomers. In comparison, MS match scores gave over 10‰ deviation and showed significant overlap in match score ranges for several isomers. This poses a risk for false positive identifications when assigning compounds based on retention time and GC-MS mass spectrum. A strategy was developed, based on Kernel Density Estimations of match scores, to construct Receiver Operating Characteristic (ROC) curves and estimate likelihood ratios (LR values) with respect to the chemical differentiation of drug related isomers. This approach, and the added value of GC-VUV is demonstrated with the chemical analysis of several samples from drug case work from the Amsterdam area involving both compounds listed in Dutch drug legislation (3,4-MDMA; 3,4-MDA; 4-MMC; 4-MEC and 4-FA) as well as their unlisted and thus uncontrolled isomers (2,3-MDMA; 2,3-MDA; 2- and 3-MMC; 2- and 3-MEC and 2- and 3-FA).

Asunto(s)

RESUMEN

Selective reporting (i.e., only reporting significant findings as opposed to all analyses or results) is a questionable research practice that undermines the integrity of published research. Psychophysiology research may be susceptible to selective reporting, given the high number of decision points and methodological complexity in analyses of psychophysiology data. We aimed to assess the presence of selective reporting and evidential value (i.e., that significant results are due to true underlying effects) in recent and past psychophysiological research by utilizing p-curve analyses. Study protocols and methods were pre-registered on the Open Science Framework (OSF). P-values and the associated test statistics were extracted from articles in the most recent issue (as of January 2018) and 10-year previous counterpart issue of three major psychophysiology journals: Psychophysiology, International Journal of Psychophysiology, and Journal of Psychophysiology. Using the p-curve application, 10 primary p-curves were conducted: all recent articles, all past articles, recent articles split by journal, past articles split by journal, recent cognitive electrophysiology articles, and past cognitive electrophysiology articles. Evidential value and generally adequate average power (≥78% average power) were present in all p-curves, except those that only included articles from the Journal of Psychophysiology because of the small number of articles published in the journal. Findings provide some positive news and indicate that, generally, results were not selectively reported, and selective reporting may not be a primary issue for this sample of psychophysiological research. Future p-curve analyses examining sub-disciplines of psychophysiology are recommended.

Asunto(s)

RESUMEN

Meta-analyses suggest inhibitory control training (ICT) may be effective for reducing food intake. However, psychological research has come under scrutiny for lack of reliability. Selective reporting (only reporting significant results) is one factor contributing to the lack of reliability in published research. Therefore, estimates of food-related ICT effects may be inaccurate. We conducted p-curve analyses to assess the presence of selective reporting, evidential value, average effect size, and power in the food-related ICT literature. Extracted p-values were selected from articles included in food-related ICT meta-analyses and an updated literature search. Four p-curve analyses resulted in 'U'-shaped distributions, suggesting evidence for both a true underlying effect and selective reporting in the food-related ICT literature. Robust analyses suggested the evidence for an underlying effect was primarily driven by the smallest p-value. The average effect size from included studies was small (d = 0.04 to 0.25). Average power to detect this effect was also small (7% to 18%). Results suggest no clear support for or against a true effect for food-related ICT. Low average effect size and power across studies suggests estimated effects are likely inflated in published literature. Higher-powered, pre-registered, longitudinal food-related ICT studies are needed to test for the presence and magnitude of ICT effects.

Asunto(s)

RESUMEN

BACKGROUND: P-curve has been proposed as a statistical test of evidential value. The distributions of sets of statistically significant p-values are tested for skewness. P-curves of true effects are right-skewed, with greater density at lower p-values than higher p-values. Analyses of null effects result in a flat or left-skewed distribution. The accuracy of p-curve has not been tested using published research analyses of a null effect. We examined whether p-curve accurately rejects a set of significant p-values obtained for a nonexistent effect. METHODS: Homeopathic ultramolecular dilutions are medicinal preparations with active substances diluted beyond Avogadro's number. Such dilute mixtures are unlikely to contain a single molecule of an active substance. We tested whether p-curve accurately rejects the evidential value of significant results obtained in placebo-controlled clinical trials of homeopathic ultramolecular dilutions. RESULTS: P-curve accurately rejected the evidential value of significant results obtained in placebo-controlled clinical trials of ultramolecular dilutions. Robustness testing using alternate p-values yielded similar results. CONCLUSION: Our results suggest that p-curve can accurately detect when sets of statistically significant results lack evidential value.

RESUMEN

A substantial number of studies have been published over the last decade, claiming that transcranial direct current stimulation (tDCS) can influence performance on cognitive tasks. However, there is some skepticism regarding the efficacy of tDCS, and evidence from meta-analyses are mixed. One major weakness of these meta-analyses is that they only examine outcomes in published studies. Given biases towards publishing positive results in the scientific literature, there may be a substantial "file-drawer" of unpublished negative results in the tDCS literature. Furthermore, multiple researcher degrees of freedom can also inflate published p-values. Recently, Simonsohn, Nelson and Simmons (2014) created a novel meta-analytic tool that examines the distribution of significant p-values in a literature, and compares it to expected distributions with different effect sizes. Using this tool, one can assess whether the selected studies have evidential value. Therefore, we examined a random selection of studies that used tDCS to alter performance on cognitive tasks, and tDCS studies on working memory in a recently published meta-analysis (Mancuso et al., 2016). Using a p-curve analysis, we found no evidence that the tDCS studies had evidential value (33% power or greater), with the estimate of statistical power of these studies being approximately 14% for the cognitive studies, and 5% (what would be expected from randomly generated data) for the working memory studies. It is likely that previous tDCS studies are substantially underpowered, and we provide suggestions for future research to increase the evidential value of future tDCS studies.

Asunto(s)

RESUMEN

BACKGROUND: The tendency to selectively report "significant" statistical results (file-drawers effect) or run selective analyses to achieve "significant" results (data-dredging) has been observed in many scientific fields. Subsequently, statistically significant findings may be due to selective reporting rather than a true effect. The p-curve, a distribution of p-values from a set of studies, is used to study aspects of statistical evidence in a scientific field. The aim of this study was to assess publication bias and evidential value in oral health research. METHODS: This was a descriptive and exploratory study that analysed the p-values published in oral health literature. The National Library of Medicine catalogue was searched for journals published in English, indexed in PubMed and tagged with dentistry Medical Subject Headings (MeSH) words. Web scraping for abstracts published between 2004 and 2014 was done and all p-values extracted. A p-curve was generated from the p-values and used for analysis. Bayesian binomial analysis was used to test the proportion of the p-values on either side of the 0.05 threshold (test for publication bias) or the 0.025 threshold (test for evidential value). The tacit assumption was that significant p-values reported were the result of publication bias. RESULTS: The present study found the use of p-values in a total of 44,315 p-values published in 12,440 abstracts. Two percent of the p-values were inaccurately reported as zero or ≥1. The p-curve was right skewed, with an intriguing bi-modality. The distribution of the p-values is also unequal on either side of 0.025 and 0.045 of the p-curve. CONCLUSIONS: This study found evidence of data-dredging, publication bias and errors in the dental literature. Although the present study was conducted on abstracts, the findings highlight a subject that should be researched in future studies that would consider the various factors that may influence p-values.

Asunto(s)