RESUMEN
Background: Workplace-based assessments (WPBA) have become integral to learner-centred medical education. As previous research has linked learner engagement to WPBA implementation, this study explores residents' and faculty members' experiences with learner engagement in the normalisation of WPBA practice. Methods: Transcendental phenomenology was used as the qualitative approach, focusing on the participants' lived experiences. A semi-structured interview guide was used to interview five faculty members and five residents who had conducted WPBA. The interviews were transcribed and analysed using phenomenological data analysis. Results: Three themes were identified between learner engagement and WPBA conduct: (a) work environment, (b) roles and relationships, and (c) mutually beneficial teaching and learning. WPBA learner engagement occurred when participants interacted with each other and with the clinical setting to facilitate teaching and learning. Both participant groups reported a desire to participate in WPBA, but time constraints at times hindered participation. The residents indicated that WPBA improved their knowledge and admitted to experiencing negative emotions during the assessment. Overall, participants recognised the reciprocal benefits of WPBA participation for their professional development. Conclusion: The findings of the study suggest that learner engagement influences the use of WPBA. Consequently, it may be beneficial to consider the role of learner engagement to normalise WPBA application for teaching and learning in the clinical context.
Contexte: Les évaluations en milieu de travail (EMT) font désormais partie intégrante de l'enseignement médical centré sur l'apprenant. Des recherches antérieures ayant établi un lien entre l'engagement de l'apprenant et la mise en Åuvre de l'EMT, cette étude explore les expériences des résidents et des membres du corps professoral en matière d'engagement de l'apprenant dans la normalisation de la pratique de l'EMT. Méthodes: La phénoménologie transcendantale a été utilisée comme approche qualitative, en portant une attention particulière aux expériences vécues des participants. Un guide d'entrevue semi-structurée a été utilisé pour interroger cinq membres du corps professoral et cinq résidents qui avaient pratiqué l'EMT. Les entrevues ont été transcrites et analysées à l'aide d'une analyse phénoménologique des données. Résultats: Trois thèmes ont été identifiés entre l'engagement de l'apprenant et la conduite des EMT : (a) l'environnement de travail, (b) les rôles et les relations, et (c) l'enseignement et l'apprentissage mutuellement bénéfiques. L'engagement de l'apprenant dans le cadre de l'EMT s'est produit lorsque les participants ont interagi entre eux et avec le milieu clinique pour faciliter l'enseignement et l'apprentissage. Les deux groupes de participants ont fait part de leur désir de participer à l'EMT, mais des contraintes de temps ont parfois entravé leur participation. Les résidents ont indiqué que l'EMT avait amélioré leurs connaissances et ont admis avoir ressenti des émotions négatives lors de l'évaluation. Dans l'ensemble, les participants ont reconnu les avantages réciproques de la participation à l'EMT pour leur développement professionnel. Conclusion: Les résultats de l'étude suggèrent que l'engagement de l'apprenant influence l'utilisation de l'EMT. Par conséquent, il peut être bénéfique de considérer le rôle de l'engagement de l'apprenant pour normaliser l'application de l'EMT pour l'enseignement et l'apprentissage dans le contexte clinique.
Asunto(s)
Docentes Médicos , Internado y Residencia , Lugar de Trabajo , Humanos , Lugar de Trabajo/psicología , Docentes Médicos/psicología , Evaluación Educacional/métodos , Investigación Cualitativa , Masculino , Femenino , Aprendizaje , Entrevistas como Asunto , Competencia ClínicaRESUMEN
The teaching of physiology plays a crucial role in the education of health care professionals. However, traditional approaches to physiology classes in undergraduate health courses in Brazil often result in passive student participation. Research has shown that active methodologies are more effective in the learning process. In this study, we introduce the game "Who Am I?-Cellular Signal Transduction Edition" as an educational tool. This game follows a popular format with well-known rules and aims to enhance understanding of basic concepts related to hormones, cell signaling, and the functioning of the endocrine system. Our findings demonstrate that the game improves student knowledge and fosters enthusiasm and active engagement among participants. Additionally, student feedback has indicated a high level of appreciation for the game. By incorporating active learning strategies and a gamified approach, "Who Am I?-Cellular Signal Transduction Edition" provides a practical and enjoyable way of teaching physiology. This innovative educational tool has the potential to revolutionize physiology instruction. Demonstrating significant improvement in students' understanding, the game underscores its efficacy in enhancing knowledge acquisition and comprehension of cellular signaling and endocrine physiology topics.NEW & NOTEWORTHY We developed "Who Am I?-Cellular Signal Transduction Edition" to assist students in comprehending concepts of cellular signal transduction. This simple and cost-effective tool is perfect for educational settings with limited resources, and it encourages active learning for both small and large groups. Pre- and posttests have shown that it effectively enhances knowledge of hormonal actions and cellular signaling. Positive feedback from students emphasizes its value in reinforcing understanding and improving classroom engagement, making it a promising educational tool.
Asunto(s)
Fisiología , Aprendizaje Basado en Problemas , Humanos , Aprendizaje Basado en Problemas/métodos , Fisiología/educación , Masculino , Femenino , Transducción de Señal , Brasil , Adulto Joven , Educación de Pregrado en Medicina/métodos , Evaluación Educacional/métodosRESUMEN
OBJECTIVE: to reflect on the perspectives of adopting the Progress Test in undergraduate nursing education. METHODS: this is a reflective study, based on authors' critical thinking and supported by national and international literature on the Progress Test application in undergraduate health courses. RESULTS: the Progress Test is as a valuable teaching-learning strategy, with potential applications and benefits for students, professors, and academic management. FINAL CONSIDERATIONS: systematic nursing education assessment indicates valuable information for different stakeholders. Understanding its potential benefits, the Progress Test is presented as a strategy that can be replicated in undergraduate nursing education, either individually, by institutions, or collaboratively, by the establishment of partnerships or consortiums of institutions.
Asunto(s)
Bachillerato en Enfermería , Evaluación Educacional , Bachillerato en Enfermería/métodos , Bachillerato en Enfermería/tendencias , Bachillerato en Enfermería/normas , Humanos , Evaluación Educacional/métodos , Estudiantes de Enfermería/estadística & datos numéricos , Estudiantes de Enfermería/psicologíaRESUMEN
The objective of this issue is a theoretical approach to the disadvantages involved in taking the certification exams of medical specializations through serial case reports and analyzing them with the Classical Test Theory (TCT). The certification exams of medical specializations correspond to criterial summative exams with high consequences. Therefore, it is imperative to maintain the highest quality standards in all the processes involved in preparing the exam. At present, it has been detected that some councils of medical specialties conduct the certification exams with tests that contain serial case reports and the psychometric analysis that they conduct is supported by the TCT; however, the structure of this type of test violates the fundamental assumptions of the TCT. The violation of the fundamental assumptions of the TCT in the tests constructed through serial case reports can lead to biases or misinterpretations of the results. Due to the above, it is advisable to use other psychometric models for the analysis of this type of test, or to avoid the use of serial case reports in the certification exams of medical specialties.
Se presenta un planteamiento teórico de las desventajas de realizar los exámenes de certificación de las especializaciones médicas mediante casos clínicos seriados y analizarlos con la Teoría Clásica del Test (TCT). Los exámenes de certificación de las especializaciones médicas corresponden a exámenes sumativos criteriales de altas consecuencias, por lo tanto, es imperativo mantener los máximos estándares de calidad en todos los procesos implicados en su elaboración. Actualmente, se ha detectado que algunos consejos de especialidades médicas realizan los exámenes de certificación con pruebas que incluyen casos clínicos seriados y que el análisis psicométrico está sustentado en la TCT; sin embargo, la estructura de este tipo de pruebas infringe los supuestos fundamentales de dicha teoría. La violación de los supuestos fundamentales de la TCT en las pruebas construidas mediante casos clínicos seriados puede conducir a sesgos o interpretación errónea de los resultados. Por lo anterior, es recomendable utilizar otros modelos psicométricos para el análisis de este tipo de pruebas, o evitar el uso de los casos clínicos seriados en los exámenes de certificación de las especialidades médicas.
Asunto(s)
Certificación , Psicometría , Humanos , Evaluación Educacional/métodosRESUMEN
OBJECTIVES: The aim of this study was to compare the performance of artificial intelligence models ChatGPT-3.5, ChatGPT-4, and Google Bard in answering Physical Medicine and Rehabilitation board-style questions, assessing their capabilities in medical education and potential clinical applications. METHODS: A comparative cross-sectional study was conducted using the PMR100, an example question set for the American Board of Physical Medicine and Rehabilitation Part I exam, focusing on artificial intelligence models' ability to answer and categorize questions by difficulty. The study evaluated the artificial intelligence models and analyzed them for accuracy, reliability, and alignment with difficulty levels determined by physiatrists. RESULTS: ChatGPT-4 led with a 74% success rate, followed by Bard at 66%, and ChatGPT-3.5 at 63.8%. Bard showed remarkable answer consistency, altering responses in only 1% of cases. The difficulty assessment by ChatGPT models closely matched that of physiatrists. The study highlighted nuanced differences in artificial intelligence models' performance across various Physical Medicine and Rehabilitation subfields. CONCLUSION: The study illustrates the potential of artificial intelligence in medical education and clinical settings, with ChatGPT-4 showing a slight edge in performance. It emphasizes the importance of artificial intelligence as a supportive tool for physiatrists, despite the need for careful oversight of artificial intelligence-generated responses to ensure patient safety.
Asunto(s)
Inteligencia Artificial , Medicina Física y Rehabilitación , Humanos , Estudios Transversales , Reproducibilidad de los Resultados , Evaluación Educacional/métodos , Competencia ClínicaRESUMEN
OBJECTIVE: To report the experience of implementing the São Paulo Nursing Courses Consortium for the Progress Test. METHOD: This is an experience report of the consortium's work in Progress Test preparation and application for Public Schools of Nursing in São Paulo in 2019, 2021 and 2022, with a descriptive analysis of the work process and the results obtained. RESULTS: The consortium's activities are structured into the following stages: planning; theme review; distributing and requesting questions; professor training; question elaboration; question reception; question selection; question validation; student registration; test application; analysis and dissemination of results. A total of 57.3% of enrolled students participated. There was a predominance of questions of medium difficulty and a gradual progression in the level of discrimination of the questions, with, in 2022, 82.5% being considered adequate. FINAL CONSIDERATIONS: The consortium has allowed the test to be applied interinstitutionally, with greater scope, accuracy, and quality of questions. Through this experience, it is expected to encourage progress testing in undergraduate nursing courses in other contexts.
Asunto(s)
Evaluación Educacional , Estudiantes de Enfermería , Brasil , Humanos , Evaluación Educacional/métodos , Bachillerato en Enfermería/métodos , Educación en Enfermería/métodos , Facultades de Enfermería/organización & administraciónRESUMEN
Assessment is an essential component for all educational programs and must check competence acquirement while foster and promote learning. Progress Test (PT) is well recognized to assess cognitive knowledge, clinical reasoning and decision making in the clinical context, offering important information about the individual performance and program quality. It is widely used in Brazilian and international medical schools; however, it still has little role in assessing medical residents in Brazil. We present the experience of a PT pilot implementation in an Infectious Diseases residency program over two years. The first, second and third-year residents did four serial exams with 40 multiple choice questions (item)/each. Preceptors were trained on best practices on item writing. All the items were reviewed by a panel of experts and, after approval, included in the item bank. All participants answered a survey on their perceptions about the experience. The final score was higher for the third-year residents in all exam applications. The level of satisfaction was high among the participants, who mentioned the learning opportunity with the exam and the feedback. PT can improve residents' assessment along the training period and residents' performance should guide review and improvement of the programs.
Asunto(s)
Competencia Clínica , Evaluación Educacional , Internado y Residencia , Humanos , Evaluación Educacional/métodos , Brasil , Proyectos Piloto , Encuestas y Cuestionarios , Infectología/educación , Evaluación de Programas y Proyectos de SaludRESUMEN
BACKGROUND: Brazilian medical schools equitably divide their medical education assessments into five content areas: internal medicine, surgery, pediatrics, obstetrics and gynecology, and public health. However, this division does not follow international patterns and may threaten the examinations' reliability and validity. OBJECTIVE: To assess the reliability indices of the content areas of serial, cross-institutional progress test examinations. DESIGN AND SETTINGS: This was an analytical, observational, and cross-sectional study conducted at nine public medical schools (mainly from the state of São Paulo) with progress test examinations conducted between 2017 and 2023. METHODS: The examinations covered the areas of basic sciences, internal medicine, surgery, pediatrics, obstetrics and gynecology, and public health. We calculated reliability indices using Cronbach's α, which indicates the internal consistency of a test. We used simple linear regressions to analyze temporal trends. RESULTS: The results showed that the Cronbach's α for basic sciences and internal medicine presented lower values, whereas gynecology, obstetrics, and public health presented higher values. After changes in the number of items and the exclusion of basic sciences as a separate content area, internal medicine ranked highest in 2023. Individually, all content areas except pediatrics remained stable over time. CONCLUSIONS: Maintaining an equitable division in assessment content may lead to suboptimal results in terms of assessment reliability, especially for internal medicine. Therefore, content sampling of medical knowledge for general assessments should be reappraised.
Asunto(s)
Evaluación Educacional , Estudios Transversales , Brasil , Reproducibilidad de los Resultados , Humanos , Evaluación Educacional/métodos , Educación Médica , Facultades de Medicina/normas , Competencia Clínica/normasRESUMEN
Objective: To develop a valid and reliable scale to measure entrepreneurship competences of nursing students, by assessing the level of development of diverse entrepreneurship dimensions. Methods: An Entrepreneurship Measurement Instrument, Catalonia (IME.Cat) was constructed, by adapting two existing instruments, and a psychometric study was performed to address the validity of the content and the construct, and the reliability. The internal consistency and the discrimination capacity of the instrument's items were examined. Results: The IME.Cat scale showed a high reliability (α=0.89) for the complete set of items. The Cronbach's α value of the individual dimensions were: Problem management=0.78; Creativity=0.76; Personal confidence =0.64; and Risk acceptance =0.46. The corrected homogeneity indices for each of the item in the instrument were high (>0.40). The Confirmatory Factorial Analysis validated the proposed structure of the items according to dimension. Conclusion: The IME.Cat scale showed solid psychometric values for assessing the entrepreneurship competences of nursing students within its dimensions, which are fundamental for the professional development of nursing.
Asunto(s)
Emprendimiento , Psicometría , Estudiantes de Enfermería , Humanos , Estudiantes de Enfermería/psicología , Reproducibilidad de los Resultados , Femenino , Masculino , Encuestas y Cuestionarios/normas , Adulto , Adulto Joven , Análisis Factorial , Evaluación Educacional/métodos , España , Competencia ProfesionalRESUMEN
Objective: To evaluate the pedagogical skills of third-year nursing students at Yangzhou University (China). Methods: A multisite quasi-experimental design was used in this study. Fifty-five participants were selected by convenience sampling. The Objective Structured Teaching Evaluation (OSTE) scale was used to assess teaching skills. The evaluation included four different stages: Teaching Background Analysis (E1), Lesson Plan Presentation (E2), Mock Class (E3) and Teaching Reflection (E4). Prior to the assessment, the teachers assigned homework to the students to complete at the four stations. Results: Fifty-five nursing students with an average age of 21.3±0.7 years participated in the study, with a predominance of female students (78.2%). The highest mean score was achieved in E1 (83.1), followed by E2 and E3 (82.5 and 82.3 respectively), while the lowest mean score was found in E4 (79.6). In E3, instructors gave lower scores for class organisation, class characteristics and overall performance compared to the self-reported scores of the standardised students (p<0.05). More than 80% of the students strongly agreed and recommended the OSTE as the primary method for assessing teaching skills in the classroom. Conclusion: Deficits in teaching skills were identified in the participating students; this information will allow specific interventions to improve the situation. The OSTE instrument was a useful method for assessing the pedagogical skills of undergraduate nursing students.
Asunto(s)
Estudiantes de Enfermería , Enseñanza , Humanos , Femenino , Masculino , Adulto Joven , China , Evaluación Educacional/métodos , Bachillerato en Enfermería/métodos , Bachillerato en Enfermería/organización & administraciónRESUMEN
INTRODUCTION: Artificial intelligence tools are being progressively integrated into medicine and surgical education. Large language models, such as ChatGPT, could provide relevant feedback aimed at improving surgical skills. The purpose of this study is to assess ChatGPT´s ability to provide feedback based on surgical scenarios. METHODS: Surgical situations were transformed into texts using a neutral narrative. Texts were evaluated by ChatGPT 4.0 and 3 surgeons (A, B, C) after a brief instruction was delivered: identify errors and provide feedback accordingly. Surgical residents were provided with each of the situations and feedback obtained during the first stage, as written by each surgeon and ChatGPT, and were asked to assess the utility of feedback (FCUR) and its quality (FQ). As control measurement, an Education-Expert (EE) and a Clinical-Expert (CE) were asked to assess FCUR and FQ. RESULTS: Regarding residents' evaluations, 96.43% of times, outputs provided by ChatGPT were considered useful, comparable to what surgeons' B and C obtained. Assessing FQ, ChatGPT and all surgeons received similar scores. Regarding EE's assessment, ChatGPT obtained a significantly higher FQ score when compared to surgeons A and B (pâ¯=â¯0.019; pâ¯=â¯0.033) with a median score of 8 vs. 7 and 7.5, respectively; and no difference respect surgeon C (score of 8; pâ¯=â¯0.2). Regarding CE´s assessment, surgeon B obtained the highest FQ score while ChatGPT received scores comparable to that of surgeons A and C. When participants were asked to identify the source of the feedback, residents, CE, and EE perceived ChatGPT's outputs as human-provided in 33.9%, 28.5%, and 14.3% of cases, respectively. CONCLUSION: When given brief written surgical situations, ChatGPT was able to identify errors with a detection rate comparable to that of experienced surgeons and to generate feedback that was considered useful for skill improvement in a surgical context performing as well as surgical instructors across assessments made by general surgery residents, an experienced surgeon, and a nonsurgeon feedback expert.
Asunto(s)
Competencia Clínica , Cirugía General , Internado y Residencia , Humanos , Cirugía General/educación , Retroalimentación Formativa , Educación de Postgrado en Medicina/métodos , Evaluación Educacional/métodos , Inteligencia Artificial , Femenino , Masculino , Entrenamiento Simulado/métodos , Cirujanos/educaciónRESUMEN
The article discusses the evolution of the Brazilian College of Surgeons (CBC) specialist title exam, highlighting the importance of evaluating not only theoretical knowledge, but also the practical skills and ethical behavior of candidates. The test was instituted in 1971, initially with only the written phase, and later included the oral practical test, starting with the 13th edition in 1988. In 2022, the assessment process was improved by including the use of simulated stations in the practical test, with the aim of assessing practical and communication skills, as well as clinical reasoning, in order to guarantee excellence in the assessment of surgeons training. The aim of this study is to demonstrate the performance of candidates in the last five years of the Specialist Title Test and to compare the performance results between the different surgical training groups of the candidates. The results obtained by candidates from the various categories enrolled in the test in the 2018 to 2022 editions were analyzed. There was a clear and statistically significant difference between doctors who had completed three years of residency recognized by the Ministry of Education in relation to the other categories of candidates for the Specialist Title..
Asunto(s)
Evaluación Educacional , Brasil , Humanos , Evaluación Educacional/métodos , Competencia Clínica , Cirujanos , Factores de Tiempo , Sociedades Médicas , Especialidades Quirúrgicas/educaciónRESUMEN
BACKGROUND: The Progress Test is an individual assessment applied to all students at the same time and on a regular basis. The test was structured in the medical undergraduate education of a conglomerate of schools to structure a programmatic assessment integrated into teaching. This paper presents the results of four serial applications of the progress test and the feedback method to students. METHODS: This assessment comprises 120 items offered online by means of a personal password. Items are authored by faculty, peer-reviewed, and approved by a committee of experts. The items are classified by five major areas, by topics used by the National Board of Medical Examiners and by medical specialties related to a national Unified Health System. The correction uses the Item Response Theory with analysis by the "Rasch" model that considers the difficulty of the item. RESULTS: Student participation increased along the four editions of the tests, considering the number of enrollments. The median performances increased in the comparisons among the sequential years in all tests, except for test1 - the first test offered to schools. Between subsequent years of education, 2nd-1st; 4th-3rd and 5th-4th there was an increase in median scores from progress tests 2 through 4. The final year of undergraduate showed a limited increase compared to the 5th year. There is a consistent increase in the median, although with fluctuations between the observed intervals. CONCLUSION: The progress test promoted the establishment of regular feedback among students, teachers and coordinators and paved the road to engagement much needed to construct an institutional programmatic assessment.
Asunto(s)
Educación de Pregrado en Medicina , Evaluación Educacional , Humanos , Evaluación Educacional/métodos , Estudiantes de MedicinaRESUMEN
INTRODUCTION: Erosive tooth wear (ETW) has been gaining attention due to its high prevalence. However, ETW clinical diagnosis is difficult and may go unnoticed by many professionals. The present study aimed to develop, implement and evaluate a theoretical-practical training using active methodologies in the development of undergraduate students' skills for the diagnosis of ETW compared to the traditional teaching method. MATERIALS AND METHODS: This randomized controlled study involved two parallel groups: control group (n = 22), with learning based only on theoretical content and test group (n = 24), learning by theoretical-practical activity mediated by monitors/tutors. The theoretical class covered the current concepts of ETW, aetiology, diagnosis, use of the BEWE index, prevention and treatment. The practical training included exercises and discussions based on the diagnosis using BEWE scores of a collection of images and extracted teeth. To evaluate the efficacy of the teaching-learning methods, a theoretical multiple-choice questionnaire and a practical test using images and extracted teeth were applied. The outcome was the number of correct answers. Groups were compared by Mann-Whitney (theoretical knowledge) and T tests (practical ability in diagnosis) (p < .05). RESULTS: There was no significant difference between groups in the theoretical evaluation (p = .866). The test group showed higher ability to diagnose ETW lesions compared to the control group in the practical tests (p = .001). The performance of ETW diagnosis was similar when images were used in comparison to extracted teeth (p = .570). CONCLUSION: The practical activity associated with theoretical classes can be a promising strategy to improve the development of undergraduate students' skills in the diagnosis of ETW.
Asunto(s)
Educación en Odontología , Estudiantes de Odontología , Erosión de los Dientes , Humanos , Educación en Odontología/métodos , Erosión de los Dientes/diagnóstico , Erosión de los Dientes/prevención & control , Erosión de los Dientes/etiología , Femenino , Masculino , Competencia Clínica , Evaluación Educacional/métodos , Adulto Joven , Encuestas y CuestionariosRESUMEN
Aprendizajes en anatomía humana tradicionalmente han sido evaluados a través del reconocimiento de estructuras anatómicas. A pesar de su popularidad, pueden tener un componente subjetivo y memorístico. El Examen Práctico Objetivo Estructurado (EPOE) se ha presentado como una manera más global de evaluación. El objetivo de este estudio fue evidenciar el efecto en el rendimiento académico tras la implementación del EPOE en pasos prácticos. En el estudio 2312 estudiantes fueron divididos: Metodología Tradicional (n=1155) y Metodología EPOE (n=1157). A su vez, los estudiantes fueron identificados según carrera: Enfermería (n=1182); Fonoaudiología (n=185); Kinesiología (n=514) y Terapia Ocupacional (n=431). Se mantuvieron las condiciones de la asignatura en ambos grupos con tres evaluaciones prácticas. Se analizaron las calificaciones obtenidas. Evaluaciones 1 y 2 no presentaron diferencias entre grupos. Evaluación 3, que evaluó sistema nervioso, mostró disminución significativa de 0,5 puntos en el grupo EPOE. El factor tipo de metodología resultó ser significativo en evaluación 3 (p < 0,001; h2p = 0,029) y promedio de las 3 evaluaciones (p < 0,029; h2p = 0,002). El factor carrera resultó ser significativo para las tres evaluaciones. La interacción de ambos factores no mostró significancia estadística. El análisis post hoc mostró diferencias significativas entre estudiantes de Enfermería con las demás carreras, ya que obtuvieron mejores calificaciones en todas las evaluaciones (p < 0,05). Metodología EPOE no produjo drásticos cambios en el rendimiento académico de los estudiantes y tiene un amplio potencial de desarrollo por su naturaleza de evaluación integral no invasiva ni traumática. Se debe considerar la naturaleza de los contenidos en el diseño de la metodología de enseñanza-aprendizaje y en la metodología de evaluación, alineando a nivel microcurricular estos aspectos fundamentales de la formación de nuevos profesionales de la salud.
SUMMARY: Learning in human anatomy has traditionally been assessed by recognizing anatomical structures. Despite their popularity, they can have a subjective component. The Objective Structured Practical Examination (EPOE) has been presented as a more global way of assessment. The aim of this study was to demonstrate the effect on academic performance after the implementation of the EPOE in practical activities. In this study 2312 students were divided into Traditional Methodology (n=1155) and EPOE Methodology (n=1157). In turn, students were identified according to career: Nursing (n=1182), Speech Therapy (n=185), Physical Therapy (n=514), and Occupational Therapy (n=431). Subject conditions were maintained in both groups with three practical evaluations. The grades obtained were analyzed. Assessments 1 and 2 showed no differences between groups. Evaluation 3, which evaluated the nervous system, showed a significant decrease of 0.5 points in the EPOE group. The type of methodology factor was significant in evaluation 3 (p < 0.001; ?2p = 0.029) and the average of 3 evaluations (p < 0.029; ?2p = 0.002). The career factor was significant for all three evaluations. The interaction of both factors did not show statistical significance. The post hoc analysis showed significant differences between nursing students and the other careers since they obtained better scores in all evaluations (p < 0.05). EPOE methodology did not produce drastic changes in students' academic performance and had a broad potential for development due to its non-invasive and non-traumatic comprehensive assessment nature. The nature of the contents should be considered in the design of the teaching-learning methodology and the evaluation methodology, aligning at the micro-curricular level with these fundamental aspects of training new health professionals.
Asunto(s)
Humanos , Masculino , Femenino , Educación Médica/métodos , Evaluación Educacional/métodos , Anatomía/educación , Estudiantes del Área de la Salud , Proyectos Piloto , Rendimiento Académico , AprendizajeRESUMEN
Background: The deployment of OpenAI's ChatGPT-3.5 and its subsequent versions, ChatGPT-4 and ChatGPT-4 With Vision (4V; also known as "GPT-4 Turbo With Vision"), has notably influenced the medical field. Having demonstrated remarkable performance in medical examinations globally, these models show potential for educational applications. However, their effectiveness in non-English contexts, particularly in Chile's medical licensing examinations-a critical step for medical practitioners in Chile-is less explored. This gap highlights the need to evaluate ChatGPT's adaptability to diverse linguistic and cultural contexts. Objective: This study aims to evaluate the performance of ChatGPT versions 3.5, 4, and 4V in the EUNACOM (Examen Único Nacional de Conocimientos de Medicina), a major medical examination in Chile. Methods: Three official practice drills (540 questions) from the University of Chile, mirroring the EUNACOM's structure and difficulty, were used to test ChatGPT versions 3.5, 4, and 4V. The 3 ChatGPT versions were provided 3 attempts for each drill. Responses to questions during each attempt were systematically categorized and analyzed to assess their accuracy rate. Results: All versions of ChatGPT passed the EUNACOM drills. Specifically, versions 4 and 4V outperformed version 3.5, achieving average accuracy rates of 79.32% and 78.83%, respectively, compared to 57.53% for version 3.5 (P<.001). Version 4V, however, did not outperform version 4 (P=.73), despite the additional visual capabilities. We also evaluated ChatGPT's performance in different medical areas of the EUNACOM and found that versions 4 and 4V consistently outperformed version 3.5. Across the different medical areas, version 3.5 displayed the highest accuracy in psychiatry (69.84%), while versions 4 and 4V achieved the highest accuracy in surgery (90.00% and 86.11%, respectively). Versions 3.5 and 4 had the lowest performance in internal medicine (52.74% and 75.62%, respectively), while version 4V had the lowest performance in public health (74.07%). Conclusions: This study reveals ChatGPT's ability to pass the EUNACOM, with distinct proficiencies across versions 3.5, 4, and 4V. Notably, advancements in artificial intelligence (AI) have not significantly led to enhancements in performance on image-based questions. The variations in proficiency across medical fields suggest the need for more nuanced AI training. Additionally, the study underscores the importance of exploring innovative approaches to using AI to augment human cognition and enhance the learning process. Such advancements have the potential to significantly influence medical education, fostering not only knowledge acquisition but also the development of critical thinking and problem-solving skills among health care professionals.
Asunto(s)
Evaluación Educacional , Licencia Médica , Femenino , Humanos , Masculino , Chile , Competencia Clínica/normas , Evaluación Educacional/métodos , Evaluación Educacional/normasRESUMEN
INTRODUCTION: The development of clinical skills requires the appropriate use of self-regulated learning (SRL). Students' use of key SRL processes as they perform a clinical skill can be identified by SRL microanalysis and used to provide feedback. SRL-microanalysis feedback only on students' key SRL processes has not been previously researched for developing clinical skills. The aim of this study was to investigate whether SRL-microanalysis feedback only on students' key SRL processes can improve both their use of SRL and their clinical skill performance. METHODS: Twenty-three final year medical students with no experience in the clinical skill required for mechanical ventilation participated in this study. Key SRL processes and clinical skill performance were measured before and after SRL microanalysis feedback. RESULTS: Overall, we found an improvement in the key SRL processes of planning and monitoring of performance, with a significant difference in monitoring. We also found an increase in students' clinical skill performance. DISCUSSION: This study, which is the first in clinical skills, demonstrated that SRL microanalysis feedback only on key SRL processes can improve both students' SRL and their clinical skill performance. studies are recommended with a great number of students and across a variety of clinical skills.
Asunto(s)
Competencia Clínica , Estudiantes de Medicina , Humanos , Proyectos Piloto , Estudiantes de Medicina/psicología , Aprendizaje , Femenino , Educación de Pregrado en Medicina , Masculino , Retroalimentación , Retroalimentación Formativa , Respiración Artificial , Evaluación Educacional/métodosRESUMEN
PURPOSE: Distance learning appears to be an attractive approach to continuing education courses, but one barrier is maintaining learner engagement throughout the course. The primary aim of this research was to evaluate the operational efficacy of a private Facebook™ group (FG) in serving as a support mechanism for distance learning courses, and its impact on three fundamental dimensions: the attrition rates of participants who did not complete the course (commonly referred to as dropout rates), the rates of course completion and approval, and the overall performance of the participants. METHODS: The participants of this quasi-experimental study comprised 159 dental students and 565 dentists enrolled in an e-learning course on oral mucosal lesion diagnosis. Prior to the initiation of the course, all participants were provided with the option to join a private FG. Within this group, moderators shared motivational messages and provided reminders concerning deadlines. Moreover, participants had the opportunity to engage in interactive discussions pertaining to topics related to the course. The course itself followed a self-guided format, employing the flipped-classroom methodology, spanning a total of 50 instructional hours. In order to assess the effectiveness of the course, participants were presented with photographs illustrating 30 oral lesions and were asked to propose diagnostic hypotheses both before and after the educational intervention (pre-tests and post-tests). RESULTS: Dentists who participated in the FG exhibited a significantly lower rate of discontinuation. As for dental students, their involvement in the group was positively associated with better performance, as determined by the percentage of accurate diagnostic hypotheses (a minimum of 70% correct responses was required for their approval in the course). CONCLUSIONS: Facebook™ demonstrates promise as a supplementary pedagogical tool in distance education courses. The interactive nature of the platform has the potential to alleviate the inherent challenges of remote learning.