RESUMO

INTRODUCTION: The Fragility Index (FI) and the FI family are statistical tools that measure the robustness of randomized controlled trials (RCT) by examining how many patients would need a different outcome to change the statistical significance of the main results of a trial. These tools have recently gained popularity in assessing the robustness or fragility of clinical trials in many clinical areas and analyzing the strength of the trial outcomes underpinning guideline recommendations. However, it has not been applied to perioperative care Clinical Practice Guidelines (CPG). OBJECTIVES: This study aims to survey clinical practice guidelines in anesthesiology to determine the Fragility Index of RCTs supporting the recommendations, and to explore trial characteristics associated with fragility. METHODS AND ANALYSIS: A methodological survey will be conducted using the targeted population of RCT referenced in the recommendations of the CPG of the North American and European societies from 2012 to 2022. FI will be assessed for statistically significant and non-significant trial results. A Poisson regression analysis will be used to explore factors associated with fragility. DISCUSSION: This methodological survey aims to estimate the Fragility Index of RCTs supporting perioperative care guidelines published by North American and European societies of anesthesiology between 2012 and 2022. The results of this study will inform the methodological quality of RCTs included in perioperative care guidelines and identify areas for improvement.

Assuntos

RESUMO

BACKGROUND: Systematic reviews and data synthesis of randomised clinical trials play a crucial role in clinical practice, research, and health policy. Trial sequential analysis can be used in systematic reviews to control type I and type II errors, but methodological errors including lack of protocols and transparency are cause for concern. We assessed the reporting of trial sequential analysis. METHODS: We searched Medline and the Cochrane Database of Systematic Reviews from 1 January 2018 to 31 December 2021 for systematic reviews and meta-analysis reports that include a trial sequential analysis. Only studies with at least two randomised clinical trials analysed in a forest plot and a trial sequential analysis were included. Two independent investigators assessed the studies. We evaluated protocolisation, reporting, and interpretation of the analyses, including their effect on any GRADE evaluation of imprecision. RESULTS: We included 270 systematic reviews and 274 meta-analysis reports and extracted data from 624 trial sequential analyses. Only 134/270 (50%) systematic reviews planned the trial sequential analysis in the protocol. For analyses on dichotomous outcomes, the proportion of events in the control group was missing in 181/439 (41%), relative risk reduction in 105/439 (24%), alpha in 30/439 (7%), beta in 128/439 (29%), and heterogeneity in 232/439 (53%). For analyses on continuous outcomes, the minimally relevant difference was missing in 125/185 (68%), variance (or standard deviation) in 144/185 (78%), alpha in 23/185 (12%), beta in 63/185 (34%), and heterogeneity in 105/185 (57%). Graphical illustration of the trial sequential analysis was present in 93% of the analyses, however, the Z-curve was wrongly displayed in 135/624 (22%) and 227/624 (36%) did not include futility boundaries. The overall transparency of all 624 analyses was very poor in 236 (38%) and poor in 173 (28%). CONCLUSIONS: The majority of trial sequential analyses are not transparent when preparing or presenting the required parameters, partly due to missing or poorly conducted protocols. This hampers interpretation, reproducibility, and validity. STUDY REGISTRATION: PROSPERO CRD42021273811.

Assuntos

RESUMO

Complete and transparent reporting of randomized controlled trial publications (RCTs) is essential for assessing their credibility. We aimed to develop text classification models for determining whether RCT publications report CONSORT checklist items. Using a corpus annotated with 37 fine-grained CONSORT items, we trained sentence classification models (PubMedBERT fine-tuning, BioGPT fine-tuning, and in-context learning with GPT-4) and compared their performance. We assessed the impact of data augmentation methods (Easy Data Augmentation (EDA), UMLS-EDA, text generation and rephrasing with GPT-4) on model performance. We also fine-tuned section-specific PubMedBERT models (e.g., Methods) to evaluate whether they could improve performance compared to the single full model. We performed 5-fold cross-validation and report precision, recall, F1 score, and area under curve (AUC). Fine-tuned PubMedBERT model that uses the sentence along with the surrounding sentences and section headers yielded the best overall performance (sentence level: 0.71 micro-F1, 0.67 macro-F1; article-level: 0.90 micro-F1, 0.84 macro-F1). Data augmentation had limited positive effect. BioGPT fine-tuning and GPT-4 in-context learning exhibited suboptimal results. Methods-specific model improved recognition of methodology items, other section-specific models did not have significant impact. Most CONSORT checklist items can be recognized reasonably well with the fine-tuned PubMedBERT model but there is room for improvement. Improved models can underpin the journal editorial workflows and CONSORT adherence checks.

Assuntos

RESUMO

Background: Multiple sclerosis (MS) is the most common non-traumatic disabling disease affecting young adults. A definitive curative treatment is currently unavailable. Many randomized controlled trials (RCTs) have reported the efficacy of Chinese herbal medicine (CHM) on MS. Because of the uncertain quality of these RCTs, the recommendations for routine use of CHM for MS remain inconclusive. The comprehensive evaluation of the quality of RCTs of CHM for MS is urgent. Methods: Nine databases, namely, PubMed, Embase, Web of Science, Cochrane Library, EBSCO, Sinomed, Wanfang Database, China National Knowledge Infrastructure, and VIP Database, were searched from inception to September 2023. RCTs comparing CHM with placebo or pharmacological interventions for MS were considered eligible. The Consolidated Standards of Reporting Trials (CONSORT) and its extension for CHM formulas (CONSORT-CHM Formulas) checklists were used to evaluate the reporting quality of RCTs. The risk of bias was assessed using the Cochrane Risk of Bias tool. The selection criteria of high-frequency herbs for MS were those with cumulative frequency over 50% among the top-ranked herbs. Results: A total of 25 RCTs were included. In the included RCTs, 33% of the CONSORT items and 21% of the CONSORT-CHM Formulas items were reported. Eligibility title, sample size calculation, allocation concealment, randomized implementation, and blinded description in CONSORT core items were reported by less than 5% of trials. For the CONSORT-CHM Formulas, the source and authentication method of each CHM ingredient was particularly poorly reported. Most studies classified the risk of bias as "unclear" due to insufficient information. The top five most frequently used herbs were, in order, Radix Rehmanniae Preparata, Radix Rehmanniae Recens, Herba Epimedii, Scorpio, and Poria. No serious adverse effect had been reported. Conclusions: The low reporting of CONSORT items and the unclear risk of bias indicate the inadequate quality of RCTs in terms of reporting completeness and result validity. The CONSORT-CHM Formulas appropriately consider the unique characteristics of CHM, including principles, formulas, and Chinese medicinal substances. To improve the quality of RCTs on CHM for MS, researchers should adhere more closely to CONSORT-CHM Formulas guidelines and ensure comprehensive disclosure of all study design elements.

Assuntos

Assuntos

Assuntos

Assuntos

RESUMO

OBJECTIVE: To evaluate the reporting quality of randomized controlled trials (RCTs) of acupuncture for depression. METHODS: Systematic searches were performed in PubMed, the Cochrane Library, EMbase, CNKI, Wanfang, SinoMed and VIP Database for RCTs of acupuncture in treatment of depression. The search time was from the establishment of database to December 1, 2023, and the language restriction was Chinese and English. The reporting quality of RCTs of acupuncture for depression was evaluated using the CONSORT statement, the international standardization for trial reporting, STRICTA, the international standard for clinical trial interventions of acupuncture, and SHARE, the guideline and checklist for reporting sham acupuncture controls. RESULTS: According to the CONSORT statement items, the items with the reporting rate less than 50% was accounted for 54.05% of all of the items for Chinese articles, and there were 8 and 1 items with a reporting rate of 0% and 100%, respectively. For the English articles, the items with the reporting rate less than 50% was accounted for 35.14% of all of the items, and there were 3 and 7 items with a reporting rate of 0% and 100%, respectively. The reporting rate of 15 items in Chinese and English articles was greater than 50%, e.g. structured abstract, background and purpose. Based on STRICTA criteria, the reporting rate of either Chinese or English articles was relatively high. The items for Chinese articles with the reporting rate less than 50% was accounted for 23.53% of all of the items, and there were 1 and 4 items with a reporting rate of 0% and 100%, respectively. For English articles, the items with the reporting rate less than 50% was accounted for 11.76% of all of the items, and there was 1 item with a reporting rate of either 0% or 100%. In compliance with SHARE checklist, the reporting rate was low for either Chinese or English articles. The items with the reporting rate less than 50% was accounted for 57.89% of all of the items for Chinese articles, and there were 2 and 0 items with a reporting rate of 0% and 100%, respectively. For English articles, the items with the reporting rate less than 50% was accounted for 52.63% of all of the items, and there was 1 item with a reporting rate of 0% and 100%, respectively. CONCLUSION: The overall reporting quality of RCTs of acupuncture for depression is low currently. It is urgent to enhance the reporting of the details on sham acupuncture control especially. It is suggested that RCTs should be reported strictly in compliance with the CONSORT statement, STRICTA criteria, and SHARE checklist in the future.

Assuntos

RESUMO

BACKGROUND: The American Academy of Orthopaedic Surgeons (AAOS) has developed a clinical practice guideline (CPG) for management of anterior cruciate ligament (ACL) injuries. Primary studies such as randomized controlled trials (RCTs) are cited as evidence for the guidelines. Given the influence that these trials have on patient care, adherence to standardized protocols for conducting and reporting RCTs is essential. PURPOSE: To evaluate the CONSORT (Consolidated Standards of Reporting Trials) Extension for Harms-related reporting of RCTs cited as supporting evidence for the AAOS CPG on the management of ACL tears. STUDY DESIGN: Cross-sectional study. METHODS: The reference section of the AAOS guideline for ACL tears was first screened for RCTs cited in the CPG. Next, each RCT was evaluated for adherence to the CONSORT Extension for Harms checklist. Both identification of RCTs and assessment of adherence were performed in a masked and duplicate process. Descriptive statistics were used to summarize adherence to CONSORT Extension for Harms items. A Pearson correlation test was conducted to assess the relationship between the year of publication and adherence to CONSORT harms reporting. RESULTS: The sample included 113 RCTs, of which 16 (14.2%) were published before the CONSORT Extension for Harms was implemented in 2004. Sample sizes ranged from 24 to 4564 participants, with a mean of 228. None of the included RCTs included all 18 items in the CONSORT Extension for Harms checklist. The mean number of checklist items reported was 4 (of 18; 22.2%). A moderate, positive, and statistically significant correlation was found between the RCT publication year and the adherence with reporting of the CONSORT Extension for Harms (t111 = 3.54; P < .001) (r = 0.32; 95% CI, 0.14-0.47). CONCLUSION: Harms were infrequently reported in RCTs cited as supporting evidence in the AAOS CPG for the management of ACL tears. One encouraging finding was the positive correlation between the year when RCTs were published and how well they adhered to reporting harms. Efforts to improve adverse event reporting are warranted, as RCTs are commonly used to make clinical decisions in orthopaedic surgery.

Assuntos

RESUMO

BACKGROUND: This study addresses the need for improved transparency and reproducibility in randomized clinical trials (RCTs) within the field of physical activity (PA) interventions. Despite efforts to promote these practices, there is limited evidence on the adherence to established reporting and methodological standards in published RCTs. The research, part of the Strengthening the Evidence in Exercise Sciences Initiative (SEES Initiative) in 2020, assessed the methodological standards and reporting quality of RCTs focusing on PA interventions. METHODS: RCTs of PA advice or exercise interventions published in 2020 were selected. Monthly searches were conducted on PubMed/MEDLINE targeting six top-tier exercise science journals. Assessments were conducted by two independent authors, based on 44 items originally from CONSORT and TIDieR reporting guidelines. These items were divided into seven domains: transparency, completeness, participants, intervention, rigor methodology, outcomes and critical analysis. Descriptive analysis was performed using absolute and relative frequencies, and exploratory analysis was done by comparing proportions using the χ2 test (α = 0.05). RESULTS: Out of 1,766 RCTs evaluated for eligibility, 53 were included. The median adherence to recommended items across the studies was 30 (18-44) items in individual assessments. Notably, items demonstrating full adherence were related to intervention description, justification, outcome measurement, effect sizes, and statistical analysis. Conversely, the least reported item pertained to mentioning unplanned modifications during trials, appearing in only 11.3% of studies. Among the 53 RCTs, 67.9% reported having a registration, and these registered studies showed higher adherence to assessed items compared to non-registered ones. CONCLUSIONS: In summary, while critical analysis aspects were more comprehensively described, aspects associated with transparency, such as protocol registrations/modifications and intervention descriptions, were reported suboptimally. The findings underscore the importance of promoting resources related to reporting quality and transparent research practices for investigators and editors in the exercise sciences discipline.

Assuntos

RESUMO

BACKGROUND: Science integrity initiatives require specific recommendations for randomised clinical trials (RCT). OBJECTIVE: To prepare a set of statements for RCT integrity through an international multi-stakeholder consensus. METHODS: The consensus was developed via multi-country multidisciplinary stakeholder group composition and engagement; evidence synthesis of 55 systematic reviews concerning RCT integrity; anonymised two-round modified Delphi survey with consensus threshold based on the average percent of majority opinions; and, a final consensus development meeting. RESULTS: There were 30 stakeholders representing 15 countries from 5 continents including trialists, ethicists, methodologists, statisticians, consumer representative, industry representative, systematic reviewers, funding body panel members, regulatory experts, authors, journal editors, peer-reviewers and advisors for resolving integrity concerns. Delphi survey response rate was 86.7% (26/30 stakeholders). There were 111 statements (73 stakeholder-provided, 46 systematic review-generated, 8 supported by both) in the initial long list, with 8 additional statements provided during the consensus rounds. Through consensus the final set consolidated 81 statements (49 stakeholder-provided, 41 systematic review-generated, 9 supported by both). The entire RCT life cycle was covered by the set of statements including general aspects (n=6), design and approval (n=11), conduct and monitoring (n=19), reporting of protocols and findings (n=20), post-publication concerns (n=12), and future research and development (n=13). CONCLUSION: Implementation of this multi-stakeholder consensus statement is expected to enhance RCT integrity.

Assuntos

Assuntos

RESUMO

BACKGROUND: Whiplash-associated disorders are a common sequela of road traffic accidents. Exercise therapy is considered an effective intervention, and it is recommended for the management of such condition. However, the application of research findings to everyday clinical practice is dependent on sufficient details being reported. OBJECTIVES: To explore the quality of reporting in studies investigating the effectiveness of exercise for whiplash-associated disorders. METHODS: A literature search was conducted to identify studies testing the effectiveness of exercise for whiplash-associated disorders. Two reporting checklists were used to evaluate reporting completeness. The median positive scores for each study and overall percentage of positive scores for each item were calculated. Percentage agreement and the Cohen's Kappa coefficient were calculated. RESULTS: Twenty-one studies were included. According to the Template for Intervention Description and Replication checklist, items were reported appropriately with a median of 29% (range 0-95%, IQR 40.5). The median number of adequately reported items per study was 5 (range 1-10, IQR 3). For the Consensus on Exercise Reporting Template checklist, items were reported appropriately with a median of 29% (range 0-57%, IQR 29). The median number of adequately reported items per study was 4 (range 0-16, IQR 8). Percentage agreement ranged from 57% to 100% while Cohen's Kappa from -0.17 to 1.00. CONCLUSIONS: The study reveals significant gaps in the quality of reporting in studies investigating exercise for whiplash-associated disorders as both checklists showed a median reporting adequacy of only 29%. Overall, the inter-rater agreement for both checklists was acceptable.

Assuntos

Assuntos

Assuntos

RESUMO

OBJECTIVES: To record the proportion of data sharing reporting in terms of primary data and/or statistical code of Randomized Controlled Trials (RCTs), published across 12 high impact journals in Dentistry, covering 6 specialty domains. Associations with certain journal, publication and outcome characteristics were examined. Transparency indicators such as registration or funding statements were assessed. METHODS: We identified and included all RCTs published from January 1st, 2017 to December 31st, 2023 in journals of high impact of the following domains: Periodontology, Endodontics, Restorative Dentistry/Prosthodontics, Orthodontics, Pediatric Dentistry, Oral/Maxillofacial Surgery. The primary outcome was the proportion of RCTs reporting their intent to share or openly shared primary data and we tested for associations with potential predictors. Funding, registration, and statistical code/script sharing practices were also examined. RESULTS: A total of 752 RCTs were included, of which only 119 (15.8%) either openly provided their data or included a statement of intention to share upon request. Only one study openly provided the statistical code underlying the analysis used. RCTs in periodontology more frequently included statements about positive intent to share (57/210;27.1%), followed by Orthodontics (35/157;22.3%). Significant effects of year, dentistry domain and continent of authorship on data sharing practices were identified (p < 0.001 in all cases). There was evidence that registered RCTs had 2.04 times higher odds for intention to share data (95%confidence interval: 1.06, 3.92;p = 0.03). CONCLUSIONS: Overall, in oral health RCTs, empirical evidence suggested very low prevalence of positive data sharing practices. Enhancing transparency is pivotal in promoting reproducibility and credibility of research findings. CLINICAL SIGNIFICANCE: The findings of this empirical report bring attention to key transparency indicators in randomized controlled trials. These largely impact on the credibility and reproducibility of the evidence base for clinical decision making.

Assuntos

Assuntos

Assuntos

Assuntos

RESUMO

Importance: Optimal health care delivery, both now and in the future, requires a continuous loop of knowledge generation, dissemination, and uptake on how best to provide care, not just determining what interventions work but also how best to ensure they are provided to those who need them. The randomized clinical trial (RCT) is the most rigorous instrument to determine what works in health care. However, major issues with both the clinical trials enterprise and the lack of integration of clinical trials with health care delivery compromise medicine's ability to best serve society. Observations: In most resource-rich countries, the clinical trials and health care delivery enterprises function as separate entities, with siloed goals, infrastructure, and incentives. Consequently, RCTs are often poorly relevant and responsive to the needs of patients and those responsible for care delivery. At the same time, health care delivery systems are often disengaged from clinical trials and fail to rapidly incorporate knowledge generated from RCTs into practice. Though longstanding, these issues are more pressing given the lessons learned from the COVID-19 pandemic, heightened awareness of the disproportionate impact of poor access to optimal care on vulnerable populations, and the unprecedented opportunity for improvement offered by the digital revolution in health care. Four major areas must be improved. First, especially in the US, greater clarity is required to ensure appropriate regulation and oversight of implementation science, quality improvement, embedded clinical trials, and learning health systems. Second, greater adoption is required of study designs that improve statistical and logistical efficiency and lower the burden on participants and clinicians, allowing trials to be smarter, safer, and faster. Third, RCTs could be considerably more responsive and efficient if they were better integrated with electronic health records. However, this advance first requires greater adoption of standards and processes designed to ensure health data are adequately reliable and accurate and capable of being transferred responsibly and efficiently across platforms and organizations. Fourth, tackling the problems described above requires alignment of stakeholders in the clinical trials and health care delivery enterprises through financial and nonfinancial incentives, which could be enabled by new legislation. Solutions exist for each of these problems, and there are examples of success for each, but there is a failure to implement at adequate scale. Conclusions and Relevance: The gulf between current care and that which could be delivered has arguably never been wider. A key contributor is that the 2 limbs of knowledge generation and implementation-the clinical trials and health care delivery enterprises-operate as a house divided. Better integration of these 2 worlds is key to accelerated improvement in health care delivery.

Assuntos