RESUMO
We assessed several agreement coefficients applied in 2x2 contingency tables, which are commonly applied in research due to dichotomization. Here, we not only studied some specific estimators but also developed a general method for the study of any estimator candidate to be an agreement measurement. This method was developed in open-source R codes and it is available to the researchers. We tested this method by verifying the performance of several traditional estimators over all possible configurations with sizes ranging from 1 to 68 (total of 1,028,789 tables). Cohen's kappa showed handicapped behavior similar to Pearson's r, Yule's Q, and Yule's Y. Scott's pi, and Shankar and Bangdiwala's B seem to better assess situations of disagreement than agreement between raters. Krippendorff's alpha emulates, without any advantage, Scott's pi in cases with nominal variables and two raters. Dice's F1 and McNemar's chi-squared incompletely assess the information of the contingency table, showing the poorest performance among all. We concluded that Cohen's kappa is a measurement of association and McNemar's chi-squared assess neither association nor agreement; the only two authentic agreement estimators are Holley and Guilford's G and Gwet's AC1. The latter two estimators also showed the best performance over the range of table sizes and should be considered as the first choices for agreement measurement in contingency 2x2 tables. All procedures and data were implemented in R and are available to download from Harvard Dataverse https://doi.org/10.7910/DVN/HMYTCK.
Assuntos
Dissidências e Disputas , Humanos , Variações Dependentes do Observador , Reprodutibilidade dos TestesRESUMO
Objetivo: validar los procedimientos de examen que establece la Guía de Práctica Clínica propuesta para el tamizaje de trastornos visuales en edad pediátrica a nivel de las consultas de cuidados de salud. Métodos: se realizó un estudio descriptivo, prospectivo y transversal en una muestra de 633 pacientes de 3 a 5 años de edad, en consulta de Oftalmología del Hospital Pediátrico Marfán, de febrero del año 2014 a enero de 2015. Participaron dos Especialistas en Pediatría de Atención Primaria y una Licenciada en Psicología, quienes realizaron el tamizaje establecido en la Guía de Práctica Clínica; dos especialistas en Oftalmología: uno como observador profesional, quien supervisó y controló que se cumpliera la aplicación del cribaje con el protocolo establecido, y el otro -junto con una Licenciada en Óptica y Optometría- realizó las pruebas seleccionadas como criterios de verdad. El estudio de la capacidad predictiva de los procedimientos de examen de la Guía de Práctica Clínica propuesta se realizó a partir de cuatro variables: agudeza visual, estereopsis, alineación ocular y examen del reflejo de fondo de ojo. Los datos se procesaron utilizando el software Statistic y Epidat 4. Resultados: se alcanzaron globalmente valores de sensibilidad de 88,15 por ciento; especificidad 97,21 por ciento, valor predictivo positivo 79,07 por ciento, valor predictivo negativo 98,57 por ciento y fuerza de concordancia muy buena, con K= 0,813. Conclusiones: los resultados obtenidos hacen fiable que la Guía de Práctica Clínica propuesta es un instrumento válido para ser recomendada en la pesquisa de trastornos visuales en las condiciones de la sociedad cubana en este grupo de edad(AU)
Objective: to validate the examination procedures established by the Clinical Practice Guideline suggested for the screening of visual disorders at pediatric ages in the primary health care. Methods: a descriptive, prospective and cross-sectional study of a sample of 633 patients aged 3-5 years was conducted at Marfan teaching pediatric hospital from February 2014 to January 2015. Two primary health care pediatricians and a psychologist performed the screening set by the clinical practice guideline; one ophthalmologist supervised and controlled the application of screening in line with the set protocol whereas the other, aided by an optometrist, made the selected tests as truth criteria. The study of the predictive capacity of the exam procedures in the suggested clinical practice guideline was based on four variables, i.e. visual acuity, stereopsis, eye alignment and fundus reflex examination. Data processing used Statistic software and Epidat 4. Results: the overall values for sensitivity was 88.15 percent, specificity 9.21 percent, positive and negative predictable values reached 79.07 percent and 98.57 percent, respectively. The agreement force was very good, with K= 0.813. Conclusions: the results confirm that the suggested Clinical Practice Guideline should be recommended as a valid tool to detect visual disorders at pediatric ages under the present conditions of the Cuban society(AU)