RESUMEN

BACKGROUND: There is some evidence for systematic biases and failures of research integrity in the anaesthesia literature. However, the features of problematic trials and effect of editorial selection on these issues have not been well quantified. METHODS: We analysed 209 randomised controlled trials submitted to Anaesthesia between 8 March 2019 and 31 March 2020. We evaluated the submitted manuscript, registry data and the results of investigations into the integrity of the trial undertaken at the time of submission. Trials were labelled 'concerning' if failures of research integrity were found, and 'problematic' if identified issues would have warranted retraction if they had been found after publication. We investigated how 'problematic' trials were detected, the distribution of p values and the risk of outcome reporting bias and p-hacking. We also investigated whether there were any factors that differed in problematic trials. RESULTS: We found that false data was the most common reason for a trial to be labelled as 'concerning', which occurred in 51/62 (82%) cases. We also found that while 195/209 (93%) trials were preregistered, we found adequate registration for only 166/209 (79%) primary outcomes, 100/209 (48%) secondary outcomes and 11/209 (5%) analysis plans. We also found evidence for a step decrease in the frequency of p values > 0.05 compared with p values < 0.05. 'Problematic' trials were all single-centre and appeared to have fewer authors (incident risk ratio (95%CI) 0.8 (0.7-0.9)), but could not otherwise be distinguished reliably from other trials. CONCLUSIONS: Identification of 'problematic' trials is frequently dependent on individual patient data, which is often unavailable after publication. Additionally, there is evidence of a risk of outcome reporting bias and p-hacking in submitted trials. Implementation of alternative research and editorial practices could reduce the risk of bias and make identification of problematic trials easier.

RESUMEN

Introduction Research integrity is an active adherence to the ethical principles and professional standards essential for the responsible practice of research. Research or scientific misconduct stands like child abuse today. The survey of National Institutes of Health (NIH)-funded scientists calculated an absolute minimum of 2325 incidents of scientific misconduct per year. A report has also shown that Iran (6.60), India (5.68), Turkey (5.38), South Korea (3.59), and China (2.00) had higher ratios of publication misconduct to distrust data or interpretations than other countries. Hence, to determine the knowledge, attitude, and practices (KAPs) of the research integrity/scientific misconduct among the faculty and postgraduates working in the medical colleges in North Karnataka (NK) and Central India (CI), this study has been carried out. Methods It is a web-based, cross-sectional study carried out with the use of Google Forms (Google, Mountain View, California). A pretested, unstructured questionnaire consisting of 25 questions was posted in the way of a link to the faculty and postgraduates working in various disciplines within the colleges of NK and CI either by using an e-mail or other social platforms like WhatsApp. Institutional Ethics Committee approval was obtained in both regions before conducting the survey. Results A total of 146 participants responded to the e-questionnaire posted to them. Participants from CI displayed better awareness in several areas compared to NK. Citing articles and/ or materials that have not been read is the common questionable research practice (QRP) they have come across, as mentioned by participants in both groups. Discussion The study reveals a moderate level of knowledge and variable attitudes toward research integrity. The "publish or perish" culture is a major contributor to misconduct. Training and awareness programs are needed to enhance ethical research practices. Conclusion This study highlights the need for improved education and policy implementation to uphold research integrity in medical colleges, emphasizing the role of academic culture in shaping ethical research practices.

RESUMEN

Research integrity is guided by a set of principles to ensure research reliability and rigor. It serves as a pillar to uphold society's trust in science and foster scientific progress. However, over the past 2 decades, a surge in research integrity concerns, including fraudulent research, reproducibility challenges, and questionable practices, has raised critical questions about the reliability of scientific outputs, particularly in biomedical research. In the biomedical sciences, any breaches in research integrity could potentially lead to a domino effect impacting patient care, medical interventions, and the broader implementation of healthcare policies. Addressing these breaches requires measures such as rigorous research methods, transparent reporting, and changing the research culture. Institutional support through clear guidelines, robust training, and mentorship is crucial to fostering a culture of research integrity. However, structural and institutional factors, including research incentives and recognition systems, play an important role in research behavior. Therefore, promoting research integrity demands a collective effort from all stakeholders to maintain public trust in the scientific community and ensure the reliability of science. Here we discuss some definitions and principles, the implications for biomedical sciences, and propose actionable steps to foster research integrity.

Asunto(s)

RESUMEN

Research Integrity (RI) is high on the agenda of both institutions and science policy. The European Union as well as national ministries of science have launched ambitious initiatives to combat misconduct and breaches of research integrity. Often, such initiatives entail attempts to regulate scientific behavior through guidelines that institutions and academic communities can use to more easily identify and deal with cases of misconduct. Rather than framing misconduct as a result of an information deficit, we instead conceptualize Questionable Research Practices (QRPs) as attempts by researchers to reconcile epistemic and social forms of uncertainty in knowledge production. Drawing on previous literature, we define epistemic uncertainty as the inherent intellectual unpredictability of scientific inquiry, while social uncertainty arises from the human-made conditions for scientific work. Our core argument-developed on the basis of 30 focus group interviews with researchers across different fields and European countries-is that breaches of research integrity can be understood as attempts to loosen overly tight coupling between the two forms of uncertainty. Our analytical approach is not meant to relativize or excuse misconduct, but rather to offer a more fine-grained perspective on what exactly it is that researchers want to accomplish by engaging in it. Based on the analysis, we conclude by proposing some concrete ways in which institutions and academic communities could try to reconcile epistemic and social uncertainties on a more collective level, thereby reducing incentives for researchers to engage in misconduct.

Asunto(s)

RESUMEN

The recent scandal involving Prof. Schillaci has raised concerns about the integrity of the Italian academic landscape, highlighting potential deeper issues within the research ecosystem. Despite the existence of comprehensive guidelines for research integrity set by the National Council of Research (CNR) and some prominent universities, the emphasis on educating research personnel about the importance of research integrity remains lacking. Additionally, prevalent issues such as nepotism and the manipulation of metrics for career advancement pose further challenges to fostering a fair and accountable research environment. While certain legislative measures have been implemented to address these issues, their effectiveness remains limited, allowing unethical practices to persist. To address these challenges, a concerted effort at the national, institutional, and individual levels is necessary. By taking these steps, Italy has the opportunity to strengthen its research ethics landscape and move toward a more transparent and ethical academic environment.

RESUMEN

In the research integrity literature, funding plays two different roles: it is thought to elevate questionable research practices (QRPs) due to perverse incentives, and it is a potential actor to incentivize research integrity standards. Recent studies, asking funders, have emphasized the importance of the latter. However, the perspective of active researchers on the impact of competitive research funding on science has not been explored yet. Here, I address this issue by conducting a series of group sessions with researchers in two different countries with different degrees of competition for funding, from three scientific fields (medical sciences, natural sciences, humanities), and in two different career stages (permanent versus temporary employment). Researchers across all groups experienced that competition for funding shapes science, with many unintended negative consequences. Intriguingly, these consequences had little to do with the type of QRPs typically being presented in the research integrity literature. Instead, the researchers pointed out that funding could result in predictable, fashionable, short-sighted, and overpromising science. This was seen as highly problematic: scientists experienced that the 'projectification' of science makes it more and more difficult to do any science of real importance: plunging into the unknown or addressing big issues that need a long-term horizon to mature. They also problematized unintended negative effects from collaboration and strategizing. I suggest it may be time to move away from a focus on QRPs in connection with funding, and rather address the real problems. Such a shift may then call for entirely different types of policy actions.

Asunto(s)

RESUMEN

We conducted a systematic review of conference papers in social psychology at two large psychology conferences in Japan: the Japanese Psychological Association and the Japanese Society for Social Psychology. The conference papers were effectively not subjected to peer review; hence, they were suitable for testing if psychologists selectively reported statistically significant findings without pressure from journal editors and reviewers. We investigated the distributions of z-values converted from the p-values reported in the articles presented at the 2013 and 2018 conferences. The z-curve analyses suggest the existence of selective reporting by the authors in 2013. The expected discovery rate (EDR) was much lower than the observed discovery rate (ODR; 7% vs. 76%, respectively), and the 95% confidence interval (CI) did not include the ODR. However, this does not mean that the set of studies completely lacked evidential value. The expected replication rate (ERR) was 31%; this is significantly higher than 5%, which was expected under the null hypothesis of no effect. Changes were observed between 2013 and 2018. The ERR increased (31% to 44%), and the EDR almost doubled (7% to 13%). However, the estimation of the maximum false discovery rate (FDR; 68% in 2013 and 35% in 2018) suggested that a substantial proportion of the reported findings were false positives. Overall, while social psychologists in Japan engaged in selective reporting, this does not mean that the entire field was covered with false positives. In addition, slight signs of improvement were observed in how they reported their findings. Still, the evidential value of the target studies was weak, even in 2018, allowing for no optimism.

Asunto(s)

RESUMEN

How good is the science in the Acceptance and Commitment Therapy (ACT) program? This article examines ACT philosophy, theory, and research on five dimensions: (1) the quality of its meta-science; (2) the clarity of its constructs; (3) the psychometrics of its principal measures; (4) the adequacy of its account of values; and (5) the quality of its research. Significant problems are found in each dimension, and suggestions for improvements are offered. ACT aligns with a Machiavellianism that is problematic in accurately describing these commitments and constituting a meta-stance that permits problematic values to be embraced. Relatedly, there is evidence of a positive bias in ACT research that has been ignored methodologically and in summaries of ACT. These problems justify significant skepticism regarding any claims from the science associated with ACT. Avoiding questionable research practices, psychometrically problematic measures, and research designs that weaken valid causal inference is recommended. Finally, an increased commitment to open science, intellectual humility, and severe testing is recommended.

Asunto(s)

RESUMEN

Research in the last decade has expressed considerable optimism about the clinical potential of psychedelics for the treatment of mental disorders. This optimism is reflected in an increase in research papers, investments by pharmaceutical companies, patents, media coverage, as well as political and legislative changes. However, psychedelic science is facing serious challenges that threaten the validity of core findings and raise doubt regarding clinical efficacy and safety. In this paper, we introduce the 10 most pressing challenges, grouped into easy, moderate, and hard problems. We show how these problems threaten internal validity (treatment effects are due to factors unrelated to the treatment), external validity (lack of generalizability), construct validity (unclear working mechanism), or statistical conclusion validity (conclusions do not follow from the data and methods). These problems tend to co-occur in psychedelic studies, limiting conclusions that can be drawn about the safety and efficacy of psychedelic therapy. We provide a roadmap for tackling these challenges and share a checklist that researchers, journalists, funders, policymakers, and other stakeholders can use to assess the quality of psychedelic science. Addressing today's problems is necessary to find out whether the optimism regarding the therapeutic potential of psychedelics has been warranted and to avoid history repeating itself.

RESUMEN

The goal of industrial/organizational (IO) psychology, is to build and organize trustworthy knowledge about people-related phenomena in the workplace. Unfortunately, as with other scientific disciplines, our discipline may be experiencing a "crisis of confidence" stemming from the lack of reproducibility and replicability of many of our field's research findings, which would suggest that much of our research may be untrustworthy. If a scientific discipline's research is deemed untrustworthy, it can have dire consequences, including the withdraw of funding for future research. In this focal article, we review the current state of reproducibility and replicability in IO psychology and related fields. As part of this review, we discuss factors that make it less likely that research findings will be trustworthy, including the prevalence of scientific misconduct, questionable research practices (QRPs), and errors. We then identify some root causes of these issues and provide several potential remedies. In particular, we highlight the need for improved research methods and statistics training as well as a re-alignment of the incentive structure in academia. To accomplish this, we advocate for changes in the reward structure, improvements to the peer review process, and the implementation of open science practices. Overall, addressing the current "crisis of confidence" in IO psychology requires individual researchers, academic institutions, and publishers to embrace system-wide change.

Asunto(s)

RESUMEN

Several recent news stories have alarmed many politicians and members of the public by reporting that indoor air pollution from gas stoves causes about 13% of childhood asthma in the United States. Research on the reproducibility and trustworthiness of epidemiological risk assessments has identified a number of common questionable research practices (QRPs) that should be avoided to draw sound causal conclusions from epidemiological data. Examples of such QRPs include claiming causation without using study designs or data analyses that allow valid causal inferences; generalizing or transporting risk estimates based on data for specific populations, time periods, and locations to different ones without accounting for differences in the study and target populations; claiming causation without discussing or quantitatively correcting for confounding, external validity bias, or other biases; and not mentioning or resolving contradictory evidence. We examine the recently estimated gas stove-childhood asthma associations from the perspective of these QRPs and conclude that it exemplifies all of them. The quantitative claim that about 13% of childhood asthma in the United States could be prevented by reducing exposure to gas stove pollution is not supported by the data collected or by the measures of association (Population Attributable Fractions) used to analyze the data. The qualitative finding that reducing exposure to gas stove pollution would reduce the burden of childhood asthma in the United States has no demonstrated validity. Systematically checking how and whether QRPs have been addressed before reporting or responding to claims that everyday exposures cause substantial harm to health might reduce social amplification of perceived risks based on QRPs and help to improve the credibility and trustworthiness of published epidemiological risk assessments.

RESUMEN

BACKGROUND: Reports of questionable or detrimental research practices (QRPs) call into question the reliability of scientific evidence and the trustworthiness of research. A critical component of the research ecosystem is the organization within which research takes place. We conducted a survey to explore the attitudes and beliefs of European and American researchers about the organisations in which they work, their own research practices and their attitudes towards research integrity and research integrity policies. METHODS: We administered an online survey (International Research Integrity Survey (IRIS)) to 2,300 active researchers based in the US and 45,000 in Europe (including UK, Norway, Iceland and Switzerland).  We employed a stratified probability sample of the authors of research articles published between 2016 and 2020 included in Clarivate's Web of Science citation database. Coverage includes researchers in the humanities, social sciences, natural sciences and medical sciences, who hold at least a master's level degree. RESULTS: In comparison to researchers in the US, European researchers admit to more QRPs and are less confident in maintaining high research integrity (RI) standards. In the US and Europe, many researchers judge their organization to fall short of best RI practice. All researchers recognize the benefits of RI, reliable knowledge and the trust of colleagues and the public, and there is support for RI training particularly among Europeans. CONCLUSION: To create and maintain a culture of integrity in scientific research, a collective commitment from researchers, their institutions and funders is needed. Researchers rely on many channels of communication about research integrity and thus the involvement of many different participants in the research system is required to make improvements. Policies must be developed to reinforce best practice rather than being seen as an irrelevance to the real business of research.

Asunto(s)

RESUMEN

Collaborative efforts to directly replicate empirical studies in the medical and social sciences have revealed alarmingly low rates of replicability, a phenomenon dubbed the 'replication crisis'. Poor replicability has spurred cultural changes targeted at improving reliability in these disciplines. Given the absence of equivalent replication projects in ecology and evolutionary biology, two inter-related indicators offer the opportunity to retrospectively assess replicability: publication bias and statistical power. This registered report assesses the prevalence and severity of small-study (i.e., smaller studies reporting larger effect sizes) and decline effects (i.e., effect sizes decreasing over time) across ecology and evolutionary biology using 87 meta-analyses comprising 4,250 primary studies and 17,638 effect sizes. Further, we estimate how publication bias might distort the estimation of effect sizes, statistical power, and errors in magnitude (Type M or exaggeration ratio) and sign (Type S). We show strong evidence for the pervasiveness of both small-study and decline effects in ecology and evolution. There was widespread prevalence of publication bias that resulted in meta-analytic means being over-estimated by (at least) 0.12 standard deviations. The prevalence of publication bias distorted confidence in meta-analytic results, with 66% of initially statistically significant meta-analytic means becoming non-significant after correcting for publication bias. Ecological and evolutionary studies consistently had low statistical power (15%) with a 4-fold exaggeration of effects on average (Type M error rates = 4.4). Notably, publication bias reduced power from 23% to 15% and increased type M error rates from 2.7 to 4.4 because it creates a non-random sample of effect size evidence. The sign errors of effect sizes (Type S error) increased from 5% to 8% because of publication bias. Our research provides clear evidence that many published ecological and evolutionary findings are inflated. Our results highlight the importance of designing high-power empirical studies (e.g., via collaborative team science), promoting and encouraging replication studies, testing and correcting for publication bias in meta-analyses, and adopting open and transparent research practices, such as (pre)registration, data- and code-sharing, and transparent reporting.

Asunto(s)

RESUMEN

Research data mismanagement (RDMM) is a serious threat to accountability, reproducibility, and re-use of data. In a recent article in this journal, it was argued that RDMM can take two forms: intentional research misconduct or unintentional questionable research practice (QRP). I disagree because the scale for severity of consequences of research misbehavior is not bimodal. Furthermore, intentionality is difficult to prove beyond doubt and is only one of many criteria that should be taken into account when deciding on the severity of a breach of research integrity and whether a sanction is justified. Making a distinction between RDMM that is research misconduct and RDMM which not puts too much emphasis on intentionality and sanctioning. The focus should rather be on improving data management practices by preventive actions, in which research institutions should take a leading role.

RESUMEN

Scholars need to be able to trust each other, because otherwise they cannot collaborate and use each other's findings. Similarly trust is essential for research to be applied for individuals, society or the natural environment. The trustworthiness is threatened when researchers engage in questionable research practices or worse. By adopting open science practices, research becomes transparent and accountable. Only then it is possible to verify whether trust in research findings is justified. The magnitude of the issue is substantial with a prevalence of four percent for both fabrication and falsification, and more than 50% for questionable research practices. This implies that researchers regularly engage in behaviors that harm the validity and trustworthiness of their work. What is good for the quality and reliability of research is not always good for a scholarly career. Navigating this dilemma depends on how virtuous the researcher at issue is, but also on the local research climate and the perverse incentives in the way the research system functions. Research institutes, funding agencies and scholarly journals can do a lot to foster research integrity, first and foremost by improving the quality of peer review and reforming researcher assessment.

RESUMEN

In many research fields, the widespread use of questionable research practices has jeopardized the credibility of scientific results. One of the most prominent questionable research practices is p-hacking. Typically, p-hacking is defined as a compound of strategies targeted at rendering non-significant hypothesis testing results significant. However, a comprehensive overview of these p-hacking strategies is missing, and current meta-scientific research often ignores the heterogeneity of strategies. Here, we compile a list of 12 p-hacking strategies based on an extensive literature review, identify factors that control their level of severity, and demonstrate their impact on false-positive rates using simulation studies. We also use our simulation results to evaluate several approaches that have been proposed to mitigate the influence of questionable research practices. Our results show that investigating p-hacking at the level of strategies can provide a better understanding of the process of p-hacking, as well as a broader basis for developing effective countermeasures. By making our analyses available through a Shiny app and R package, we facilitate future meta-scientific research aimed at investigating the ramifications of p-hacking across multiple strategies, and we hope to start a broader discussion about different manifestations of p-hacking in practice.

RESUMEN

Questionable research practices (QRP) are actions taken by researchers that span a range of concern related to violation of research best practices, and ultimately expose institutions and research participants to risk. Numerous studies have shown that QRP are common. The continued prevalence of QRP indicates that existing approaches for dealing with QRP are falling short. In this editorial we discuss the risks associated with QRP and propose mitigation strategies at the institutional level using a common QRP as an example, questionable treatment of subgroup analyses. We argue that the need for institutional intervention in cases such as this are particularly motivating when both the investigator and the institution have a substantial financial conflict of interest related to intellectual property that requires the investigator's expertise to continue developing. To address this, we propose an expansion of the traditional conflict of interest management process.

Asunto(s)

RESUMEN

Publication bias and p-hacking are two well-known phenomena that strongly affect the scientific literature and cause severe problems in meta-analyses. Due to these phenomena, the assumptions of meta-analyses are seriously violated and the results of the studies cannot be trusted. While publication bias is very often captured well by the weighting function selection model, p-hacking is much harder to model and no definitive solution has been found yet. In this paper, we advocate the selection model approach to model publication bias and propose a mixture model for p-hacking. We derive some properties for these models, and we compare them formally and through simulations. Finally, two real data examples are used to show how the models work in practice.

Asunto(s)

RESUMEN

Good record keeping practice and research data management underlie responsible research conduct and promote reproducibility of research findings in the sciences. In many cases of research misconduct, inadequate research data management frequently appear as an accompanying finding. Findings of disorganized or otherwise poor data archival or loss of research data are, on their own, not usually considered as indicative of research misconduct. Focusing on the availability of raw/primary data and the replicability of research based on these, we posit that most, if not all, instances of research data mismanagement (RDMM) could be considered a questionable research practice (QRP). Furthermore, instances of RDMM at their worst could indeed be viewed as acts of research misconduct. Here, we analyze with postulated scenarios the contexts and circumstances under which RDMM could be viewed as a significant misrepresentation of research (ie. falsification), or data fabrication. We further explore how RDMM might potentially be adjudicated as research misconduct based on intent and consequences. Defining how RDMM could constitute QRP or research misconduct would aid the formulation of relevant institutional research integrity policies to mitigate undesirable events stemming from RDMM.