RESUMEN

OBJECTIVES: Quantitative bias analysis (QBA) methods evaluate the impact of biases arising from systematic errors on observational study results. This systematic review aimed to summarize the range and characteristics of QBA methods for summary-level data published in the peer-reviewed literature. STUDY DESIGN AND SETTING: We searched MEDLINE, Embase, Scopus, and Web of Science for English-language articles describing QBA methods. For each QBA method, we recorded key characteristics, including applicable study designs, bias(es) addressed; bias parameters, and publicly available software. The study protocol was preregistered on the Open Science Framework (https://osf.io/ue6vm/). RESULTS: Our search identified 10,249 records, of which 53 were articles describing 57 QBA methods for summary-level data. Of the 57 QBA methods, 53 (93%) were explicitly designed for observational studies, and 4 (7%) for meta-analyses. There were 29 (51%) QBA methods that addressed unmeasured confounding, 19 (33%) misclassification bias, 6 (11%) selection bias, and 3 (5%) multiple biases. Thirty-eight (67%) QBA methods were designed to generate bias-adjusted effect estimates and 18 (32%) were designed to describe how bias could explain away observed findings. Twenty-two (39%) articles provided code or online tools to implement the QBA methods. CONCLUSION: In this systematic review, we identified a total of 57 QBA methods for summary-level epidemiologic data published in the peer-reviewed literature. Future investigators can use this systematic review to identify different QBA methods for summary-level epidemiologic data. PLAIN LANGUAGE SUMMARY: Quantitative bias analysis (QBA) methods can be used to evaluate the impact of biases on observational study results. However, little is known about the full range and characteristics of available methods in the peer-reviewed literature that can be used to conduct QBA using information reported in manuscripts and other publicly available sources without requiring the raw data from a study. In this systematic review, we identified 57 QBA methods for summary-level data from observational studies. Overall, there were 29 methods that addressed unmeasured confounding, 19 that addressed misclassification bias, six that addressed selection bias, and three that addressed multiple biases. This systematic review may help future investigators identify different QBA methods for summary-level data.

RESUMEN

PURPOSE: HIV pre-exposure prophylaxis (PrEP) may increase rates of bacterial sexually transmitted infections (STIs) among gay, bisexual, and other men who have sex with men (GBM) through risk compensation (eg, an increase in condomless sex or number of partners); however, longitudinal studies exploring the time-dependent nature of PrEP uptake and bacterial STIs are limited. We used marginal structural models to estimate the effect of PrEP uptake on STI incidence. METHODS: We analyzed data from the iCruise study, an online longitudinal study of 535 Ontarian GBM from July 2017 to April 2018, to estimate the effects of PrEP uptake on incidence of self-reported bacterial STIs (chlamydia, gonorrhea, and syphilis) collected with 12 weekly diaries. The incidence rate was calculated as the number of infections per 100 person-months, with evaluation of the STIs overall and individually. We used marginal structural models to account for time-varying confounding and quantitative bias analysis to evaluate the sensitivity of estimates to nondifferential outcome misclassification. RESULTS: Participating GBM were followed up for a total of 1,623.5 person-months. Overall, 70 participants (13.1%) took PrEP during the study period. Relative to no uptake, PrEP uptake was associated with an increased incidence rate of gonorrhea (incidence rate ratio = 4.00; 95% CI, 1.67-9.58), but not of chlamydia or syphilis, and not of any bacterial STI overall. Accounting for misclassification, the median incidence rate ratio for gonorrhea was 2.36 (95% simulation interval, 1.08-5.06). CONCLUSIONS: We observed an increased incidence rate of gonorrhea associated with PrEP uptake among Ontarian GBM that was robust to misclassification. Although our findings support current guidelines for integrating gonorrhea screening with PrEP services, additional research should consider the long-term impact of PrEP among this population.

RESUMEN

OBJECTIVE: We aimed to demonstrate the use of quantitative bias analysis (QBA), which reveals the effects of systematic error, including confounding, misclassification and selection bias, on study results in epidemiological studies published in the period from 2010 to mid-23. METHOD: The articles identified through a keyword search using Pubmed and Scopus were included in the study. The articles obtained from this search were eliminated according to the exclusion criteria, and the articles in which QBA analysis was applied were included in the detailed evaluation. RESULTS: It can be said that the application of QBA analysis has gradually increased over the 13-year period. Accordingly, the number of articles in which simple is used as a method in QBA analysis is 9 (9.89%), the number of articles in which the multidimensional approach is used is 10 (10.99%), the number of articles in which the probabilistic approach is used is 60 (65.93%) and the number of articles in which the method is not specified is 12 (13.19%). The number of articles with misclassification bias model is 44 (48.35%), the number of articles with uncontrolled confounder(s) bias model is 32 (35.16%), the number of articles with selection bias model is 7 (7.69%) and the number of articles using more than one bias model is 8 (8.79%). Of the 49 (53.85%) articles in which the bias parameter source was specified, 19 (38.78%) used internal validation, 26 (53.06%) used external validation and 4 (8.16%) used educated guess, data constraints and hypothetical data. Probabilistic approach was used as a bias method in 60 (65.93%) of the articles, and mostly beta (8 [13.33%)], normal (9 [15.00%]) and uniform (8 [13.33%]) distributions were selected. CONCLUSION: The application of QBA is rare in the literature but is increasing over time. Future researchers should include detailed analyzes such as QBA analysis to obtain inferences with higher evidence value, taking into account systematic errors.

RESUMEN

We test the robustness of the self-controlled risk interval (SCRI) design in a setting where time between doses may introduce time-varying confounding, using both negative control outcomes (NCOs) and quantitative bias analysis (QBA). All vaccinated cases identified from 5 European databases between 1 September 2020 and end of data availability were included. Exposures were doses 1-3 of the Pfizer, Moderna, AstraZeneca, and Janssen COVID-19 vaccines; outcomes were myocarditis and otitis externa (NCO). The SCRI used a 60-day control window and dose-specific 28-day risk windows, stratified by vaccine brand and adjusted for calendar time. The QBA included two scenarios: (i) baseline probability of the confounder was higher in the control window and (ii) vice versa. The NCO was not associated with any of the COVID-19 vaccine types or doses except Moderna dose 1 (IRR = 1.09, 95%CI 1.01-1.09). The QBA suggested even the strongest literature-reported confounder (COVID-19; RRmyocarditis = 18.3) could only explain away part of the observed effect from IRR = 3 to IRR = 1.40. The SCRI seems robust to unmeasured confounding in the COVID-19 setting, although a strong unmeasured confounder could bias the observed effect upward. Replication of our findings for other safety signals would strengthen this conclusion.

RESUMEN

Purpose: To determine the contribution of recall bias to the observed excess in mental ill-health in those reporting harassment at work. Methods: A prospective cohort of 1885 workers in welding and electrical trades was contacted every six months for up to 5 years, asking whether they were currently anxious or depressed and whether this was made worse by work. Only at the end of the study did we ask about any workplace harassment they had experienced at work. We elicited sensitivity and specificity of self-reported bullying from published reliability studies and formulated priors that reflect the possibility of over-reporting of workplace harassment (exposure) by those whose anxiety or depression was reported to be made worse by work (cases). We applied the resulting misclassification models to probabilistic bias analysis (PBA) of relative risks. Results: We observe that PBA implies that it is unlikely that biased misclassification due to the study subjects' states of mind could have caused the entire observed association. Indeed, the results demonstrated that doubling of risk of anxiety or depression following workplace harassment is plausible, with the unadjusted relative risk attenuated with understated uncertainty. Conclusions: It seems unlikely that risk of anxiety or depression following workplace harassment can be explained by the form of recall bias that we proposed.

RESUMEN

Quantitative bias analysis (QBA) permits assessment of the expected impact of various imperfections of the available data on the results and conclusions of a particular real-world study. This article extends QBA methodology to multivariable time-to-event analyses with right-censored endpoints, possibly including time-varying exposures or covariates. The proposed approach employs data-driven simulations, which preserve important features of the data at hand while offering flexibility in controlling the parameters and assumptions that may affect the results. First, the steps required to perform data-driven simulations are described, and then two examples of real-world time-to-event analyses illustrate their implementation and the insights they may offer. The first example focuses on the omission of an important time-invariant predictor of the outcome in a prognostic study of cancer mortality, and permits separating the expected impact of confounding bias from non-collapsibility. The second example assesses how imprecise timing of an interval-censored event - ascertained only at sparse times of clinic visits - affects its estimated association with a time-varying drug exposure. The simulation results also provide a basis for comparing the performance of two alternative strategies for imputing the unknown event times in this setting. The R scripts that permit the reproduction of our examples are provided.

RESUMEN

OBJECTIVES: The Inflation Reduction Act (IRA), enacted in 2022, brings substantial reforms to the US healthcare system, particularly regarding Medicare. A key aspect includes the introduction of Medicare price negotiation. The objective of this commentary is to explore the implications of the IRA for US pharmaceutical companies, with a specific focus on the role of real-world evidence (RWE) in the context of Medicare reforms. METHODS: This commentary uses a qualitative analysis of the IRA's provisions related to healthcare and pharmaceutical regulation, focusing on how these reforms change the evidence requirements for pharmaceutical companies. It discusses the methodological aspects of generating and using RWE, including techniques such as target trial emulation and quantitative bias analysis methods to address biases inherent in RWE. RESULTS: This commentary highlights that the IRA introduces a unique approach to value assessment in the United States by evaluating drug value several years after launch, as opposed to at launch, similar to health technology assessments in other regions. It underscores the central role of RWE in comparing drug effectiveness across diverse clinical scenarios to improve the accuracy of real-world data comparisons. Furthermore, this article identifies key methodologies for managing the inherent biases in RWE, which are crucial for generating credible evidence for IRA price negotiations. CONCLUSIONS: This article underscores the importance of these methodologies in ensuring credible evidence for IRA price negotiations. It advocates for an integrated approach in evidence generation, positioning RWE as pivotal for informed pricing discussions in the US healthcare landscape.

Asunto(s)

RESUMEN

PURPOSE: Use of real-world data (RWD) for external controls added to single-arm trials (SAT) is increasingly prevalent in regulatory submissions. Due to inherent differences in the data-generating mechanisms, biases can arise. This paper aims to illustrate how to use quantitative bias analysis (QBA). METHODS: Advanced non-small cell lung cancer (NSCLC) serves as an example, where many small subsets of patients with molecular tumor subtypes exist. First, some sources of bias that may occur in oncology when comparing RWD to SAT are described. Second, using a hypothetical immunotherapy agent, a dataset is simulated based on expert input for survival analysis of advanced NSCLC. Finally, we illustrate the impact of three biases: missing confounder, misclassification of exposure, and outcome evaluation. RESULTS: For each simulated scenario, bias was induced by removing or adding data; hazard ratios (HRs) were estimated applying conventional analyses. Estimating the bias-adjusted treatment effect and uncertainty required carefully selecting the bias model and bias factors. Although the magnitude of each biased and bias-adjusted HR appeared moderate in all three hypothetical scenarios, the direction of bias was variable. CONCLUSION: These findings suggest that QBA can provide an intuitive framework for bias analysis, providing a key means of challenging assumptions about the evidence. However, the accuracy of bias analysis is itself dependent on correct specification of the bias model and bias factors. Ultimately, study design should reduce bias, but QBA allows us to evaluate the impact of unavoidable bias to assess the quality of the evidence.

Asunto(s)

RESUMEN

Development of medicines in rare oncologic patient populations are growing, but well-powered randomized controlled trials are typically extremely challenging or unethical to conduct in such settings. External control arms using real-world data are increasingly used to supplement clinical trial evidence where no or little control arm data exists. The construction of an external control arm should always aim to match the population, treatment settings and outcome measurements of the corresponding treatment arm. Yet, external real-world data is typically fraught with limitations including missing data, measurement error and the potential for unmeasured confounding given a nonrandomized comparison. Quantitative bias analysis (QBA) comprises a collection of approaches for modelling the magnitude of systematic errors in data which cannot be addressed with conventional statistical adjustment. Their applications can range from simple deterministic equations to complex hierarchical models. QBA applied to external control arm represent an opportunity for evaluating the validity of the corresponding comparative efficacy estimates. We provide a brief overview of available QBA approaches and explore their application in practice. Using a motivating example of a comparison between pralsetinib single-arm trial data versus pembrolizumab alone or combined with chemotherapy real-world data for RET fusion-positive advanced non-small cell lung cancer (aNSCLC) patients (1-2% among all NSCLC), we illustrate how QBA can be applied to external control arms. We illustrate how QBA is used to ascertain robustness of results despite a large proportion of missing data on baseline ECOG performance status and suspicion of unknown confounding. The robustness of findings is illustrated by showing that no meaningful change to the comparative effect was observed across several 'tipping-point' scenario analyses, and by showing that suspicion of unknown confounding was ruled out by use of E-values. Full R code is also provided.

Asunto(s)

RESUMEN

BACKGROUND: Certain associations observed in the National Birth Defects Prevention Study (NBDPS) contrasted with other research or were from areas with mixed findings, including no decrease in odds of spina bifida with periconceptional folic acid supplementation, moderately increased cleft palate odds with ondansetron use and reduced hypospadias odds with maternal smoking. OBJECTIVES: To investigate the plausibility and extent of differential participation to produce effect estimates observed in NBDPS. METHODS: We searched the literature for factors related to these exposures and participation and conducted deterministic quantitative bias analyses. We estimated case-control participation and expected exposure prevalence based on internal and external reports, respectively. For the folic acid-spina bifida and ondansetron-cleft palate analyses, we hypothesized the true odds ratio (OR) based on prior studies and quantified the degree of exposure over- (or under-) representation to produce the crude OR (cOR) in NBDPS. For the smoking-hypospadias analysis, we estimated the extent of selection bias needed to nullify the association as well as the maximum potential harmful OR. RESULTS: Under our assumptions (participation, exposure prevalence, true OR), there was overrepresentation of folic acid use and underrepresentation of ondansetron use and smoking among participants. Folic acid-exposed spina bifida cases would need to have been ≥1.2× more likely to participate than exposed controls to yield the observed null cOR. Ondansetron-exposed cleft palate cases would need to have been 1.6× more likely to participate than exposed controls if the true OR is null. Smoking-exposed hypospadias cases would need to have been ≥1.2 times less likely to participate than exposed controls for the association to falsely appear protective (upper bound of selection bias adjusted smoking-hypospadias OR = 2.02). CONCLUSIONS: Differential participation could partly explain certain associations observed in NBDPS, but questions remain about why. Potential impacts of other systematic errors (e.g. exposure misclassification) could be informed by additional research.

Asunto(s)

RESUMEN

The use of treatment effects derived from nonrandomized studies (NRS) in health technology assessment (HTA) is growing. NRS carry an inherently greater risk of bias than randomized controlled trials (RCTs). Although bias can be mitigated to some extent through appropriate approaches to study design and analysis, concerns around data availability and quality and the absence of randomization mean residual biases typically render the interpretation of NRS challenging. Quantitative bias analysis (QBA) methods are a range of methods that use additional, typically external, data to understand the potential impact that unmeasured confounding and other biases including selection bias and time biases can have on the results (i.e., treatment effects) from an NRS. QBA has the potential to support HTA bodies in using NRS to support decision-making by quantifying the magnitude, direction, and uncertainty of biases. However, there are a number of key aspects of the use of QBA in HTA which have received limited discussion. This paper presents recommendations for the use of QBA in HTA developed using a multi-stakeholder workshop of experts in HTA with a focus on QBA for unmeasured confounding.

Asunto(s)

RESUMEN

BACKGROUND: Failure to appropriately account for unmeasured confounding may lead to erroneous conclusions. Quantitative bias analysis (QBA) can be used to quantify the potential impact of unmeasured confounding or how much unmeasured confounding would be needed to change a study's conclusions. Currently, QBA methods are not routinely implemented, partly due to a lack of knowledge about accessible software. Also, comparisons of QBA methods have focused on analyses with a binary outcome. METHODS: We conducted a systematic review of the latest developments in QBA software published between 2011 and 2021. Our inclusion criteria were software that did not require adaption (i.e., code changes) before application, was still available in 2022, and accompanied by documentation. Key properties of each software tool were identified. We provide a detailed description of programs applicable for a linear regression analysis, illustrate their application using two data examples and provide code to assist researchers in future use of these programs. RESULTS: Our review identified 21 programs with [Formula: see text] created post 2016. All are implementations of a deterministic QBA with [Formula: see text] available in the free software R. There are programs applicable when the analysis of interest is a regression of binary, continuous or survival outcomes, and for matched and mediation analyses. We identified five programs implementing differing QBAs for a continuous outcome: treatSens, causalsens, sensemakr, EValue, and konfound. When applied to one of our illustrative examples, causalsens incorrectly indicated sensitivity to unmeasured confounding whereas the other four programs indicated robustness. sensemakr performs the most detailed QBA and includes a benchmarking feature for multiple unmeasured confounders. CONCLUSIONS: Software is now available to implement a QBA for a range of different analyses. However, the diversity of methods, even for the same analysis of interest, presents challenges to their widespread uptake. Provision of detailed QBA guidelines would be highly beneficial.

Asunto(s)

Asunto(s)

RESUMEN

Prevalence estimates are fundamental to epidemiological studies. Although they are highly vulnerable to misclassification bias, the risk of bias assessment of prevalence estimates is often neglected. Quantitative bias analysis (QBA) can effectively estimate misclassification bias in epidemiological studies; however, relatively few applications are identified. One reason for its low usage is the lack of knowledge and tools for these methods among researchers. To expand existing evaluation methods, based on the QBA principles, three indicators are proposed. One is the relative bias that quantifies the bias direction through its signs and the bias magnitude through its quantity. The second is the critical point of positive test proportion in case of a misclassification bias that is equal to zero. The third is the bound of positive test proportion equal to adjusted prevalence at misclassification bias level α. These indicators express the magnitude, direction and uncertainty of the misclassification bias of prevalence estimates, respectively. Using these indicators, it was found that slight oscillations of the positive test proportion within a certain range can lead to substantial increases in the misclassification bias. Hence, researchers should account for misclassification error analytically when interpreting the significance of adjusted prevalence for epidemiological decision making. This highlights the importance of applying QBA to these analyses. In this article, we have used three real-world cases to illustrate the characteristics and calculation methods of presented indicators. To facilitate application, an Excel-based calculation tool is provided.

Asunto(s)

RESUMEN

BACKGROUND: Maternal age is increasingly recognized as a predictor of birth outcomes. Given the importance of birth and growth outcomes for children's development, wellbeing and survival, this study examined the effect of maternal age on infant birth and growth outcomes at 6 months and mortality. Additionally, we conducted quantitative bias analysis (QBA) to estimate the role of selection bias and unmeasured confounding on the effect of maternal age on infant mortality. METHODS: We used data from randomized-controlled trials (RCTs) of 21 555 neonates in Burkina Faso conducted in 2019-2020. Newborns of mothers aged 13-19 years (adolescents) and 20-40 years (adults) were enrolled in the study 8-27 days after birth and followed for 6 months. Measurements of child's anthropometric measures were collected at baseline and 6 months. We used multivariable linear regression to compare child anthropometric measures at birth and 6 months, and logistic regression models to obtain the odds ratio (OR) of all-cause mortality. Using multidimensional deterministic analysis, we assessed scenarios in which the difference in selection probability of adolescent and adult mothers with infant mortality at 6 months increased from 0% to 5%, 10%, 15% and 20% if babies born to adolescent mothers more often died during the first week or were of lower weight and hence were not eligible to be included in the original RCT. Using probabilistic bias analysis, we assessed the role of unmeasured confounding by socio-economic status (SES). RESULTS: Babies born to adolescent mothers on average had lower weight at birth, lower anthropometric measures at baseline, similar growth outcomes from enrolment to 6 months and higher odds of all-cause mortality by 6 months (adjusted OR = 2.17, 95% CI 1.35 to 3.47) compared with those born to adult mothers. In QBA, we found that differential selection of adolescent and adult mothers could bias the observed effect (OR = 2.24, 95% CI 1.41 to 3.57) towards the null [bias-corrected OR range: 2.37 (95% CI 1.49 to 3.77) to 2.84 (95% CI 1.79 to 4.52)], whereas unmeasured confounding by SES could bias the observed effect away from the null (bias-corrected OR: 2.06, 95% CI 1.31 to 2.64). CONCLUSIONS: Our findings suggest that delaying the first birth from adolescence to adulthood may improve birth outcomes and reduce mortality of neonates. Babies born to younger mothers, who are smaller at birth, may experience catch-up growth, reducing some of the anthropometric disparities by 6 months of age.

Asunto(s)

RESUMEN

Evidence generated from nonrandomized studies (NRS) is increasingly submitted to health technology assessment (HTA) agencies. Unmeasured confounding is a primary concern with this type of evidence, as it may result in biased treatment effect estimates, which has led to much criticism of NRS by HTA agencies. Quantitative bias analyses are a group of methods that have been developed in the epidemiological literature to quantify the impact of unmeasured confounding and adjust effect estimates from NRS. Key considerations for application in HTA proposed in this article reflect the need to balance methodological complexity with ease of application and interpretation, and the need to ensure the methods fit within the existing frameworks used to assess nonrandomized evidence by HTA bodies.

Asunto(s)

RESUMEN

Due to uncertainty regarding the potential impact of unmeasured confounding, health technology assessment (HTA) agencies often disregard evidence from nonrandomized studies when considering new technologies. Quantitative bias analysis (QBA) methods provide a means to quantify this uncertainty but have not been widely used in the HTA setting, particularly in the context of cost-effectiveness modelling (CEM). This study demonstrated the application of an aggregate and patient-level QBA approach to quantify and adjust for unmeasured confounding in a simulated nonrandomized comparison of survival outcomes. Application of the QBA output within a CEM through deterministic and probabilistic sensitivity analyses and under different scenarios of knowledge of an unmeasured confounder demonstrates the potential value of QBA in HTA.

Asunto(s)

Asunto(s)

RESUMEN

Data collected from a validation substudy permit calculation of a bias-adjusted estimate of effect that is expected to equal the estimate that would have been observed had the gold standard measurement been available for the entire study population. In this paper, we develop and apply a framework for adaptive validation to determine when sufficient validation data have been collected to yield a bias-adjusted effect estimate with a prespecified level of precision. Prespecified levels of precision are decided a priori by the investigator, based on the precision of the conventional estimate and allowing for wider confidence intervals that would still be substantively meaningful. We further present an applied example of the use of this method to address exposure misclassification in a study of transmasculine/transfeminine youth and self-harm. Our method provides a novel approach to effective and efficient estimation of classification parameters as validation data accrue, with emphasis on the precision of the bias-adjusted estimate. This method can be applied within the context of any parent epidemiologic study design in which validation data will be collected and modified to meet alternative criteria given specific study or validation study objectives.

Asunto(s)

RESUMEN

PURPOSE: To outline a method for obtaining more accurate estimates of drug use in the United States (US) general population by correcting survey data for underreported and unknown drug use. METHODS: We simulated a population (n = 100,000) reflecting the demographics of the US adult population per the 2018 American Community Survey. Within this population, we simulated the "true" and self-reported prevalence of past-month cannabis and cocaine use by using available estimates of underreporting. We applied our algorithm to samples of the simulated population to correct self-reported estimates and recover the "true" population prevalence, validating our approach. We applied this same method to 2018 National Survey on Drug Use and Health (NSDUH) data to produce a range of underreporting-corrected estimates. RESULTS: Simulated self-report sensitivities varied by drug and sampling method (cannabis: 77.6%-78.5%, cocaine: 14.3%-22.1%). Across repeated samples, mean corrected prevalences (calculated by dividing self-reported prevalence by estimated sensitivity) closely approximated simulated "true" prevalences. Applying our algorithm substantially increased 2018 NSDUH estimates (self-report: cannabis = 10.5%, cocaine = 0.8%; corrected: cannabis = 15.6%-16.6%, cocaine = 2.7%-5.5%). CONCLUSIONS: National drug use prevalence estimates can be corrected for underreporting using a simple method. However, valid application of this method requires accurate data on the extent and correlates of misclassification in the general US population.

Asunto(s)