RESUMEN
BACKGROUND: Dietary exposure assessments are a critical issue in evaluating human nutrition studies; however, nutrition-specific criteria are not consistently included in existing bias assessment tools. OBJECTIVES: Our objective was to develop a set of risk of bias (RoB) tools that integrated nutrition-specific criteria into validated generic assessment tools to address RoB issues, including those specific to dietary exposure assessment. METHODS: The Nutrition QUality Evaluation Strengthening Tools (NUQUEST) development and validation process included 8 steps. The first steps identified 1) a development strategy; 2) generic assessment tools with demonstrated validity; and 3) nutrition-specific appraisal issues. This was followed by 4) generation of nutrition-specific items and 5) development of guidance to aid users of NUQUEST. The final steps used established ratings of selected studies and feedback from independent raters to 6) assess reliability and validity; 7) assess formatting and usability; and 8) finalize NUQUEST. RESULTS: NUQUEST is based on the Scottish Intercollegiate Guidelines Network checklists for randomized controlled trials, cohort studies, and case-control studies. Using a purposive sample of 45 studies representing the 3 study designs, interrater reliability was high (Cohen's κ: 0.73; 95% CI: 0.52, 0.93) across all tools and at least moderate for individual tools (range: 0.57-1.00). The use of a worksheet improved usability and consistency of overall interrater agreement across all study designs (40% without worksheet, 80%-100% with worksheet). When compared to published ratings, NUQUEST ratings for evaluated studies demonstrated high concurrent validity (93% perfect or near-perfect agreement). Where there was disagreement, the nutrition-specific component was a contributing factor in discerning exposure methodological issues. CONCLUSIONS: NUQUEST integrates nutrition-specific criteria with generic criteria from assessment tools with demonstrated reliability and validity. NUQUEST represents a consistent and transparent approach for evaluating RoB issues related to dietary exposure assessment commonly encountered in human nutrition studies.
Asunto(s)
Sesgo , Métodos Epidemiológicos , Evaluación Nutricional , Ciencias de la Nutrición/normas , Proyectos de Investigación/estadística & datos numéricos , Lista de Verificación , Humanos , Reproducibilidad de los ResultadosRESUMEN
The aims of this study were (1) to design a user-friendly instrument to assess and optimize patient information material (PIM), (2) to develop an English version, and (3) to test its psychometric properties. The instrument was needed to optimize the top-down developed PIM of the psycho-oncological care programme isPO. First, a literature-based PIM checklist was developed by a team of patient representatives, cancer care experts and professional researchers. Next, the checklist's reliability and validity were analysed by having cancer survivors assess the initial and optimized version of the isPO-leaflet. The User-friendly Patient Information Material Checklist (UPIM-Check), developed participatorily, was found to be effective for evaluating PIM. It uses a traffic light scale, and suggestions for improvement can be given for each criterion. Its reliability appeared to be excellent (α = 0.927). The optimized leaflet was rated significantly better than the initial one. The UPIM-Check is a reliable and valid instrument, which enables end-users (e.g., patients) to assess and optimize the quality of PIM according to scientific criteria and the needs of end-users. A bottom-up approach was essential for developing and validating the UPIM-Check. End-users constantly contributed with their specific knowledge. Thus, their position as co-researchers was significantly strengthened.
Asunto(s)
Lista de Verificación , Humanos , Psicometría , Reproducibilidad de los Resultados , Encuestas y CuestionariosRESUMEN
BACKGROUND: There are currently no validated guidelines to assess the quality of the content and the delivery style of scientific podium surgical presentations. We have developed a simple, short, and reliable instrument to objectively assess the overall quality of scientific podium presentations. METHODS: A simple and efficient rating instrument was developed to assess the scientific content and presentation style/skills of the surgical residents' presentations from 1996 to 2013. Absolute and consistency agreement for the different sections of the instrument was determined and assessed overtime, by stage of the project and study design. Intraclass correlation coefficients with 95% confidence intervals were calculated and reported using a mixed-effects model. RESULTS: Inter-rater reliability for both absolute and consistency agreement was substantial for total score and for each of the 3 sections of the instrument. The absolute agreement for the overall rating of the presentations was .87 (.63 to .98) and .78 (.50 to .95), and the consistency agreement was .90 (.70 to .99) and .87 (.67 to .97) for the 2012 and 2013 institutional research presentations, respectively. Rater agreement for evaluating project stage and different study designs varied from .70 to .81 and was consistent over the years. The consistency agreement in rating of the presentation was .77 for both faculty and resident raters. CONCLUSIONS: Standardized methodological assessment of research presentations (SHARP) instrument rates the scientific quality of the research and style of the delivered presentation. It is highly reliable in scoring the quality of the all study designs regardless of their stage. We recommend that researchers focus on presenting the key concepts and significant elements of their evidence using visually simple slides in a professionally engaging manner for effective delivery of their research and better communication with the audience.