RESUMEN

In sparse data meta-analyses (with few trials or zero events), conventional methods may distort results. Although better-performing one-stage methods have become available in recent years, their implementation remains limited in practice. This study examines the impact of using conventional methods compared to one-stage models by re-analysing meta-analyses from the Cochrane Database of Systematic Reviews in scenarios with zero event trials and few trials. For each scenario, we computed one-stage methods (Generalised linear mixed model [GLMM], Beta-binomial model [BBM], Bayesian binomial-normal hierarchical model using a weakly informative prior [BNHM-WIP]) and compared them with conventional methods (Peto-Odds-ratio [PETO], DerSimonian-Laird method [DL] for zero event trials; DL, Paule-Mandel [PM], Restricted maximum likelihood [REML] method for few trials). While all methods showed similar treatment effect estimates, substantial variability in statistical precision emerged. Conventional methods generally resulted in smaller confidence intervals (CIs) compared to one-stage models in the zero event situation. In the few trials scenario, the CI lengths were widest for the BBM on average and significance often changed compared to the PM and REML, despite the relatively wide CIs of the latter. In agreement with simulations and guidelines for meta-analyses with zero event trials, our results suggest that one-stage models are preferable. The best model can be either selected based on the data situation or, using a method that can be used in various situations. In the few trial situation, using BBM and additionally PM or REML for sensitivity analyses appears reasonable when conservative results are desired. Overall, our results encourage careful method selection.

RESUMEN

BACKGROUND: Systematic reviews and meta-analysis of time-to-event outcomes are frequently published within the Cochrane Database of Systematic Reviews (CDSR). However, these outcomes are handled differently across meta-analyses. They can be analysed on the hazard ratio (HR) scale or can be dichotomized and analysed as binary outcomes using effect measures such as odds ratios (OR) or risk ratios (RR). We investigated the impact of reanalysing meta-analyses from the CDSR that used these different effect measures. METHODS: We extracted two types of meta-analysis data from the CDSR: either recorded in a binary form only ("binary"), or in binary form together with observed minus expected and variance statistics ("OEV"). We explored how results for time-to-event outcomes originally analysed as "binary" change when analysed using the complementary log-log (clog-log) link on a HR scale. For the data originally analysed as HRs ("OEV"), we compared these results to analysing them as binary on a HR scale using the clog-log link or using a logit link on an OR scale. RESULTS: The pooled HR estimates were closer to 1 than the OR estimates in the majority of meta-analyses. Important differences in between-study heterogeneity between the HR and OR analyses were also observed. These changes led to discrepant conclusions between the OR and HR scales in some meta-analyses. Situations under which the clog-log link performed better than logit link and vice versa were apparent, indicating that the correct choice of the method does matter. Differences between scales arise mainly when event probability is high and may occur via differences in between-study heterogeneity or via increased within-study standard error in the OR relative to the HR analyses. CONCLUSIONS: We identified that dichotomising time-to-event outcomes may be adequate for low event probabilities but not for high event probabilities. In meta-analyses where only binary data are available, the complementary log-log link may be a useful alternative when analysing time-to-event outcomes as binary, however the exact conditions need further exploration. These findings provide guidance on the appropriate methodology that should be used when conducting such meta-analyses.

Asunto(s)

RESUMEN

OBJECTIVES: A recent paper by Doi et al. advocated completely replacing the relative risk (RR) with the odds ratio (OR) as the effect measure in clinical trials and meta-analyses with binary outcomes. Besides some practical advantages of RR over OR, Doi et al.'s key assumption that the OR is "portable" in the meta-analysis, that is, study-specific ORs are likely not correlated with baseline risks, was not well justified. STUDY DESIGNS AND SETTINGS: We summarized Spearman's rank correlation coefficient between study-specific ORs and baseline risks in 40,243 meta-analyses from the Cochrane Database of Systematic Reviews. RESULTS: Study-specific ORs tend to be higher in studies with lower baseline risks of disease for most meta-analyses in Cochrane Database of Systematic Reviews. Using an actual meta-analysis example, we demonstrate that there is a strong negative correlation between OR (RR or RD) with the baseline risk and the conditional effects notably vary with baseline risks. CONCLUSIONS: Replacing RR or RD with OR is currently unadvisable in clinical trials and meta-analyses. It is possible that no effect measure is "portable" in a meta-analysis. In addition to the overall (or marginal) effect, we suggest presenting the conditional effect based on the baseline risk using a bivariate generalized linear mixed model.

Asunto(s)

RESUMEN

OBJECTIVE: Recently Doi et al. argued that risk ratios should be replaced with odds ratios in clinical research. We disagreed, and empirically documented the lack of portability of odds ratios, while Doi et al. defended their position. In this response we highlight important errors in their position. STUDY DESIGN AND SETTING: We counter Doi et al.'s arguments by further examining the correlations of odds ratios, and risk ratios, with baseline risks in 20,198 meta-analyses from the Cochrane Database of Systematic Reviews. RESULTS: Doi et al.'s claim that odds ratios are portable is invalid because 1) their reasoning is circular: they assume a model under which the odds ratio is constant and show that under such a model the odds ratio is portable; 2) the method they advocate to convert odds ratios to risk ratios is biased; 3) their empirical example is readily-refuted by counter-examples of meta-analyses in which the risk ratio is portable but the odds ratio isn't; and 4) they fail to consider the causal determinants of meta-analytic inclusion criteria: Doi et al. mistakenly claim that variation in odds ratios with different baseline risks in meta-analyses is due to collider bias. Empirical comparison between the correlations of odds ratios, and risk ratios, with baseline risks show that the portability of odds ratios and risk ratios varies across settings. CONCLUSION: The suggestion to replace risk ratios with odds ratios is based on circular reasoning and a confusion of mathematical and empirical results. It is especially misleading for meta-analyses and clinical guidance. Neither the odds ratio nor the risk ratio is universally portable. To address this lack of portability, we reinforce our suggestion to report variation in effect measures conditioning on modifying factors such as baseline risk; understanding such variation is essential to patient-centered practice.

Asunto(s)

Asunto(s)

RESUMEN

OBJECTIVE: Retrieving the qualitative literature can be challenging, but the number and specific choice of databases are key factors. The aim of the present study is to provide guidance for the choice of databases for retrieving qualitative health research. STUDY DESIGN AND SETTING: Seventy-one qualitative systematic reviews, from the Cochrane Database of Systematic Reviews and JBI database of Systematic Reviews and Implementation Reports, including 927 qualitative studies, were used to analyze the coverage of the qualitative literature in nine bibliographic databases. RESULTS: The results show that 94.4% of the qualitative studies are indexed in at least one database, with a lower coverage for publication types other than journal articles. Maximum recall with two databases is 89.1%, with three databases recall increases to 92% and maximum recall with four databases is 93.1%. The remaining 6.9% of the publications consists of 1.3% scattered across five databases and 5.6% that are not indexed in any of the nine databases used in this study. CONCLUSION: Retrieval in one or a few-although well selected-databases does not provide all the relevant qualitative studies. The remaining studies needs to be located using several other databases and alternative search strategies.

Asunto(s)

RESUMEN

BACKGROUND: Meta-analysis combines multiple independent studies, which can increase power and provide better estimates. However, it is unclear how best to deal with studies with zero events; such studies are also known as double-zero-event studies (DZS). Several statistical methods have been proposed, but the agreement among different approaches has not been systematically assessed using real-world published systematic reviews. METHODS: The agreement of five commonly used methods (i.e., the inverse-variance, Mantel-Haenszel, Peto, Bayesian, and exact methods) was assessed using the Cohen's κ coefficients using 368 meta-analyses with rare events selected from the Cochrane Database of Systematic Reviews. Three continuity corrections, including the correction of a constant 0.5, the treatment arm continuity correction (TACC), and the empirical (EMP) correction, were used to handle DZS when applying inverse-variance and Mantel-Haenszel methods. RESULTS: When the proportion of DZS studies was lower than 50% in a meta-analysis, different methods had moderately high agreement. However, when this proportion was increased to be over 50%, the agreement among the methods decreased to different extents. For the Bayesian, exact, and Peto methods and the inverse-variance and Mantel-Haenszel methods using the EMP correction, their agreement coefficients with the inverse-variance and Mantel-Haenszel methods using a constant 0.5 and TACC decreased from larger than 0.70 to smaller than 0.30. In contrast, the agreement coefficients only decreased slightly among the Bayesian, exact, and Peto methods and the inverse-variance and Mantel-Haenszel methods using the EMP correction. CONCLUSIONS: To utilize all available information and reduce research waste and avoid overestimating the effect, meta-analysts should incorporate DZS, rather than simply removing them. The Peto and other conventional methods with continuity correction should be avoided when the proportion of DZS is extremely high. The exact and Bayesian methods are highly recommended, except when none of the included studies have an event in one or both treatment arms.

Asunto(s)

RESUMEN

OBJECTIVES: Evaluate comparative harm rates from medical interventions in pediatric randomized clinical trials (RCTs) from more developed (MDCs) and less developed countries (LDCs). STUDY DESIGN AND SETTING: Meta-epidemiologic empirical evaluation of Cochrane Database of Systematic Reviews (June 2014) meta-analyses reporting clinically important harm-outcomes (severe adverse events [AEs], discontinuations due to AEs, any AE, and mortality) that included at least one pediatric RCT from MDCs and at least one from LDCs. We estimated relative odds ratios (RORs) for each harm, within each meta-analysis, between RCTs from MDCs and LDCs and calculated random-effects-summary-RORs (sRORs) for each harm across multiple meta-analyses. RESULTS: Only 1% (26/2,363) of meta-analyses with clinically important harm-outcomes in the entire Cochrane Database of Systematic Reviews included pediatric RCTs both from MDCs and LDCs. We analyzed 26 meta-analyses with 244 data sets from pediatric RCTs, 116 from MDCs and 128 from LDCs (64 and 66 unique RCTs respectively). The summary ROR was 0.92 (95% confidence intervals: 0.78-1.08) for severe AEs; 1.13 (0.54-2.34) for discontinuations due to AEs; 1.10 (0.77-1.59) for any AE; and 0.99 (0.61-1.61) for mortality and for the all-harms-combined-end point 0.96 (0.83-1.10). Differences of ROR-point-estimates ≥2-fold between MDCs and LDCs were identified in 35% of meta-analyses. CONCLUSION: We found no major systematic differences in harm rates in pediatric trials between MDCs and LDCs, but data on harms in children were overall very limited.

Asunto(s)

RESUMEN

OBJECTIVES: Evaluating the variation in the strength of the effect across studies is a key feature of meta-analyses. This variability is reflected by measures like τ(2) or I(2), but their clinical interpretation is not straightforward. A prediction interval is less complicated: it presents the expected range of true effects in similar studies. We aimed to show the advantages of having the prediction interval routinely reported in meta-analyses. DESIGN: We show how the prediction interval can help understand the uncertainty about whether an intervention works or not. To evaluate the implications of using this interval to interpret the results, we selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n=2009) or continuous (n=1254) outcome, and generated 95% prediction intervals for them. RESULTS: In 72.4% of 479 statistically significant (random-effects p<0.05) meta-analyses in the Cochrane Database 2009-2013 with heterogeneity (I(2)>0), the 95% prediction interval suggested that the intervention effect could be null or even be in the opposite direction. In 20.3% of those 479 meta-analyses, the prediction interval showed that the effect could be completely opposite to the point estimate of the meta-analysis. We demonstrate also how the prediction interval can be used to calculate the probability that a new trial will show a negative effect and to improve the calculations of the power of a new trial. CONCLUSIONS: The prediction interval reflects the variation in treatment effects over different settings, including what effect is to be expected in future patients, such as the patients that a clinician is interested to treat. Prediction intervals should be routinely reported to allow more informative inferences in meta-analyses.

Asunto(s)

RESUMEN

OBJECTIVES: Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. We empirically evaluated whether there is also a relationship between trial size and heterogeneity (τ). STUDY DESIGN AND SETTING: We selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n = 2,009) or continuous (n = 1,254) outcome. The association between estimated τ and trial size was evaluated across meta-analyses using regression and within meta-analyses using a Bayesian approach. Small trials were predefined as those having standard errors (SEs) over 0.2 standardized effects. RESULTS: Most meta-analyses were based on few (median 4) trials. Within the same meta-analysis, the small study τS(2) was larger than the large-study τL(2) [average ratio 2.11; 95% credible interval (1.05, 3.87) for dichotomous and 3.11 (2.00, 4.78) for continuous meta-analyses]. The imprecision of τS was larger than of τL: median SE 0.39 vs. 0.20 for dichotomous and 0.22 vs. 0.13 for continuous small-study and large-study meta-analyses. CONCLUSION: Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.

Asunto(s)