RESUMEN

The establishment of a reasonable teacher evaluation indicator system has always been a research hotspot in teacher evaluation. Simplifying and essential indicators are the foundation for maintaining the speed and accuracy of teacher evaluation. Therefore, optimizing indicators in a convincing and interpretable manner becomes highly important. Due to the complex causal relationships and fuzzy uncertainties among teacher evaluation indicators, this paper proposes a method that combines the Triangular Fuzzy Decision-making Trial and Evaluation Laboratory Model (Fuzzy-DEMATEL) with Backpropagation Neural Network (BP) to optimize the complexity of assessment systems and identify key indicators, thereby establishing a precise and rational teacher evaluation index system. DEMATEL allows us to simplify and analyze the complex causal relationships among assessment indicators, mapping them into a causal relationship diagram. Fuzzy logic effectively handles the fuzzy and uncertain relationships among the indicators, converting fuzzy information into computable forms. The BP neural network is a data training model that, from an objective data perspective, compensates for subjective errors, thereby optimizing our indicator results. In addition, we conducted empirical and comparative research using relevant data from the TIMSS 2019 dataset, and found that this method can reduce the original indicator quantity by approximately 28 %-30 %, compared to methods such as Multi-Criteria Decision Making (MCDM), the results are superior and the indicators are more accurate.

RESUMEN

The fast-paced evolution of technology has compelled the digitalization of education, requiring educators to interact with computers and develop digital competencies relevant to the teaching-learning process. This need has prompted various organizations to define frameworks for assessing digital competency emphasizing teachers' interaction with computer technologies in education. Different authors have presented assessment methods for teachers' digital competence based on the video analysis of recorded classes using sensors such as cameras, microphones, or electroencephalograms. The main limitation of these solutions is the large number of resources they require, making it difficult to assess large numbers of teachers in resource-constrained environments. This article proposes the automation of teachers' digital competence evaluation process based on monitoring metrics obtained from teachers' interaction with a Learning Management System (LMS). Based on the Digital Competence Framework for Educators (DigCompEdu), indicators were defined and extracted that allow automatic measurement of a teacher's competency level. A tool was designed and implemented to conduct a successful proof of concept capable of automating the evaluation process of all university faculty, including 987 lecturers from different fields of knowledge. Results obtained allow for drawing conclusions on technological adoption according to the teacher's profile and planning educational actions to improve these competencies.

RESUMEN

OBJECTIVES: The right intuition for self-assessment without overestimating or underestimating oneself compared to theoretical knowledge is a skill that requires conscious practice and is very often paradoxically opposed to the level of knowledge. Self-assessment is an essential skill for dental professionals for lifelong learning and improvement. The objective of our study was to assess the correlation between theoretical knowledge and the difference between self- and teaching doctor-assessment. METHODS: The app "digital course organizer" for organization and assessment was used for the self- and teaching doctor-assessment of students for each day of patient treatment during the clinical courses at a university hospital. The difference between the self- and teaching doctor-assessment was compared to the score of an initial theoretical written test at the beginning of the eighth semester to assess the correlation between overestimation or underestimation and theoretical knowledge. RESULTS: A total of 309 dental students over four semesters in the clinical study phase (fourth and fifth years; eighth and 10th semesters) participated in the investigation. The overall view of all values showed very low correlations (<0.2) of the assessment difference values of both practical courses for the initial written test. CONCLUSION: There were very low correlations between the initial written test (theoretical knowledge) and the difference between self- and teaching doctor-assessment as well as no evidence of overestimation and underestimation.

Asunto(s)

RESUMEN

The Algerian Ministry of Higher Education announced the closure of universities and switched to distance teaching due to the strong and rapid spread of COVID-19. The pandemic that forced this change created many obstacles to pedagogy. In fact, many related issues had to be resolved and other alternative teaching strategies examined. This article defines a strategy that ensures the pedagogical continuity of laboratory practical work of computer graphics course in a COVID- 19 context. It also includes data analysis of student satisfaction questionnaire on the lab work strategy quality. As a result of this research, the success rates in the graphics design course reached 91,79% and students' grades ranged from 4.33/20 to 18.17/20. This result is consistent with the combined-type graphical representation of COLLES. Guidelines and best practices are provided for performing online learning labs, including the university's online learning platform as well as different teaching scenarios, tutoring and peer assessment techniques. Computer science instructors can use this innovative strategy to develop content-specific courses, establish recommended skills, and deploy technology to improve their teaching methods.

RESUMEN

BACKGROUND: After the Corona pandemic, medical education has shifted to virtual education, but there has been limited time and possibilities for empowering faculty for this purpose. Therefore, it seems necessary to evaluate the quality of the provided training and provide feedback to the faculty in order to improve the quality of training. The purpose of this study was to investigate the effect of teacher formative evaluation by peer observation method on the quality of virtual teaching of basic medical sciences faculty. METHODS: In this study, seven trained faculty members observed and based on a checklist evaluated the quality of 2 virtual sessions taught by each faculty of basic medical sciences, and provided them feedback; after at least 2 weeks, their Virtual teachings were again observed and evaluated. The results before and after providing feedback were compared through SPSS software. RESULTS: After intervention, significant improvements were observed in the average scores of "overall virtual performance", "virtual classroom management" and "content quality". Specifically, there was a significant increase in the average score of "overall virtual performance" and "virtual class management" among female faculty, and the average score of "overall virtual performance" among permanently employed faculty members with more than 5 years of teaching experience, before and after intervention (p < 0.05). CONCLUSION: Virtual and online education can be a suitable platform for the implementation of formative and developmental model of peer observation of faculty; and should be considered as an opportunity to empower and improve the quality of the faculty' performance in virtual education.

Asunto(s)

RESUMEN

The practice of educational research by rural teachers is highly valued and very important for their professional development and for the revitalization of rural education. This study explored the components of educational research activities among rural teachers (Study 1). Based on the results, a regional norm for Hunan province was formulated, and discriminant criteria were developed for the evaluation of educational research skills and accomplishments among rural teachers (Study 2). In Study 1, data from 892 Chinese rural teachers working at compulsory education schools in Hunan Province (a representative province in central China), divided into two samples, were found to support the constructs included in the measurement instrument. Exploratory and confirmatory factor analyses of the 33 items of the Rural Teachers' Educational Research Self-rating Scale identified a first-order model with three factors: educational research on basic educational activities (BEA), educational research involving the creation of an educational community (CEC), and educational research involving the refinement and popularization of educational theory (RPE). Based on the results of Study 1, in Study 2, a set of norms for educational research skills and accomplishments among rural teachers was formulated based on the data from Hunan Province. This norm can serve as a reference standard for the evaluation of rural teachers' educational research skills and accomplishments. The components of rural teachers' educational research activities are discussed, and suggestions for the formulation of education policies are provided.

RESUMEN

In this study, we examined the quality of Puhui early education services perceived by Chinese teachers with evidence from a national validation study. Firstly, using a stratified cluster sampling method, the Puhui Early Education Service Quality Scale (PEESQ) was developed and validated with 994 Chinese teachers. Secondly, the psychometric properties indicated that PEESQ is a reliable and valid scale with three dimensions: service adequacy, service balance, and public service welfare. Thirdly, descriptive statistics found that Chinese teachers generally appreciated the quality of Puhui services, ranking the construct of "public welfare" the highest and "service balance" the lowest. Finally, hierarchical regression analysis found that urbanicity positively predicted teacher evaluations after controlling for demographic and kindergarten factors. The findings imply that more funding support and policy attention should be provided to help enhance the quality of Puhui early education services (EES). Moreover, this study extends the framework of research on teacher evaluation of EES and provides useful tools to evaluate universal EES in China and beyond.

Asunto(s)

RESUMEN

To develop a reflective tool for promoting teacher effectiveness among preservice EFL teachers, this article seeks to investigate the effectiveness degree of an EFL teacher as it is being depicted in a popular TV show. This is conducted using the Stronge teacher evaluation system to assess the main character's level of effectiveness as an EFL teacher (Mr. Brown) in the Mind Your Language TV show. Drawing upon the intersection of the character's effectiveness within the framework of characterization, representation, and perceived realism theory, a qualitative research method involving seven performance indicators of the Stronge teacher evaluation system was adopted to assess the main character. The findings showed that the character is a highly effective EFL teacher because his personal and professional characteristics are evidenced in the evaluation system's seven performance standards. This suggests that the Show can be used as a reflective tool by preservice EFL teachers to construct and enhance their professional identities and instructional practices. This work contributes to the existing knowledge of teachers' representation in movies and its implication in preservice EFL education by providing the first study on the representation of EFL teachers in a TV show. A future research direction is also presented.

RESUMEN

Resumen La transformación reciente de la Universidad latinoamericana ha sido definida por las necesidades y premisas del modelo neoliberal. En este artículo tratamos algunos de sus impactos sociales en Colombia por medio del análisis de las lógicas que tales transformaciones han implicado para las universidades privadas; en particular, la gestión académica, la contratación y evaluación docente, así como la producción de conocimiento. Puesto que se trata de un modelo universitario en función del mercado, las estrategias de control epistemológico en las instituciones privadas delimitan no solo las posibilidades de vincular la investigación a sectores sociales subalternos, sino también la promoción de conocimientos considerados improductivos. Como límites corporativos al conocimiento crítico del país, tales impactos no solo dan cuenta de los efectos del modelo neoliberal, sino que han sido una de las estrategias para su profundización.

Abstract The recent transformation of the Latin American university has been defined by the needs and premises of the neoliberal model. In this article, we discuss some of its social impacts in Colombia through the analysis of the logics that such transformations have implied for private universities; in particular, academic management, teacher recruitment, and teacher evaluation, as well as knowledge production. Since it is a market-driven university model, the strategies of epistemological control, in private institutions, limit not only the possibilities of linking research to subaltern social sectors, but also the promotion of knowledge considered unproductive. As corporate limits to critical knowledge in the country, such impacts not only account for the effects of the neoliberal model, but also have been one of the strategies for its deepening.

RESUMEN

BACKGROUND: Student evaluation is an essential component in feedback processes in faculty and learner development. Ease of use and low cost have made paper evaluation forms a popular method within teaching programmes, but they are often seen as a formality, offering variable value towards the improvement of teaching. Students report poor motivation to engage with existing feedback tools whilst teachers describe receiving vague, contradicting, or irrelevant information. We believe that feedback for teachers needs to be a two-way process, similar to feedback for students, for it to be effective. An online feedback tool has been implemented for third-year medical students from Imperial College London to promote open discussion between teachers and students. The feedback tool is accessible throughout students' clinical attachment with the option of maintaining anonymity. We aim to explore the benefits and challenges of this online feedback tool and assess its value as a method for teacher feedback. METHODS: Qualitative data was obtained from both volunteer third-year medical students of Imperial College London and Clinical Teaching Fellows using three focus groups and a questionnaire. Data was analysed through iterative coding and thematic analysis to provide over-arching analytical themes. RESULTS: Twenty-nine students trialled this feedback tool with 17 responding to the evaluative questionnaire. Four over-arching themes were identified: reasons for poor participation with traditional feedback tools; student motivators to engage with 'open feedback'; evaluative benefits from open feedback; concerns and barriers with open feedback. CONCLUSION: This feedback tool provides a platform for two-way feedback by encouraging open, transparent discussion between teachers and learners. It gives a unique insight into both teachers and peers' perspectives. Students engage better when their responses are acknowledged by the teachers. We elaborate on the benefits and challenges of public open feedback and approaches to consider in addressing the self-censorship of critical comments.

Asunto(s)

RESUMEN

:: Entrustable professional activities (EPAs) provide a novel approach to support teachers' structured professionalization and to assess improvement in teaching competence thereafter. Despite their novelty, it is important to assess EPAs as a construct to ensure that they accurately reflect the work of the targeted profession. BACKGROUND:: The co-creation of an EPA framework for training and entrustment of small-group facilitators has been discussed in the literature. Although a rigorous design process was used to develop the framework, its content validity has not been established yet.Approach: A modified Delphi technique was used. Three survey rounds were conducted from December 2019 to April 2020. Expert health professions educationalists were recruited using purposive sampling and snowball techniques. In Round 1, a rubric consisting of seven items was used to assess the quality of nine pre-designed EPAs. In Round 2, competencies required to perform the agreed-upon EPAs were selected from 12 competencies provided. In Round 3, consensus was sought on sub-activities recommended for agreed-upon EPAs. Quantitative data were analyzed using multiple statistical analyses, including item-wise and rubric-wise content validity indices, asymmetric confidence interval, mean, standard deviation, and response frequencies. Qualitative data were thematically analyzed using content analysis. FINDINGS:: Three of the nine proposed EPAs achieved statistical consensus for retention. These EPAs were: (1) preparing an activity, (2) facilitating a small-group session, and (3) reflecting upon self and the session. Nine of the 12 pre-determined competencies achieved consensus and were then mapped against each agreed-upon EPA based on their relevance. Finally, results indicated consensus on five, six, and four sub-activities for EPA 1, EPA 2, and EPA 3, respectively. CONCLUSIONS:: The final framework delineates three EPAs for small-group facilitation and their associated sub-activities. The full description of each EPA provided in this article includes the title, context, task specification, required competencies, and entrustment resources. Program developers, administrative bodies, and teaching staff may find this EPA framework useful to structure faculty development, to entrust teachers, and to support personal development.

Asunto(s)

RESUMEN

Student assessment nonparticipation (or opt out) has increased substantially in K-12 schools in states across the country. This increase in opt out has the potential to impact achievement and growth (or value-added) measures used for educator and institutional accountability. In this simulation study, we investigated the extent to which value-added measures of teacher quality are affected as a result of varying degrees of opt out, as well as a result of various types of nonrandom opt out. Results show that the magnitude of opt out and choice of classification scheme has a greater impact on value-added estimates than the type of opt-out patterns simulated in this study. Specifically, root mean square differences in value-added estimates increased as magnitude of opt out increased. In addition, teacher effectiveness classification agreement decreased as opt out magnitude increased. One type of opt out, where the highest achieving students in the highest achieving classrooms opted out, had the largest impact on stability than the other types of opt outs.

RESUMEN

Background: Recent reports suggest that faculty development (FD) programs need a structured framework to design training and assess improvement in teaching performance of participants. Entrustable professional activities (EPAs) can serve as a novel framework to plan and conduct structured FD programs, and to assess the proficiency of small group facilitators after training. Objective: The researchers aimed to develop an EPAs framework for small group facilitators. Design: In March 2019, three workshops were organized to develop the EPAs framework by using a participatory action design approach. An orientation workshop was conducted to train the participating students and teachers. Then, a design workshop was conducted to develop the EPA framework, where data were collected from three sources: scribe notes, audio recordings, and field charts. Thematic analysis was performed, and consensus was sought from participants on the extracted professional tasks and competencies in the consensus workshop. In the third workshop, the participants also mapped professional tasks with relevant competencies. Results: A total of 15 teachers and 15 studentsf participated in the co-design process. Through a robust thematic analysis of multisource data, 57 professional tasks and 52 competencies emerged, which were converged into 11 tasks and 17 competencies after removing duplicating and non-qualifying professional tasks and competencies. Finally, a consensus was achieved on nine tasks and 12 competencies. Conclusions: The proposed EPAs framework can serve as a road map for longitudinal training and entrustment of small group facilitators. It can also guide small group facilitators in their continuous professional development and in building their teaching portfolios.

Asunto(s)

RESUMEN

The present study explores the convergent and predictive validity for several widely used measures of teaching quality from the Measures of Effective Teaching Project (Bill and Melinda Gates Foundation, 2009-2011). Specifically, the Classroom Assessment Scoring System (CLASS; Pianta, Hamre, & Mintz, 2012), the Framework for Teaching (FFT; Danielson Group, 2013), and the Tripod Student Perceptions Scale (Tripod; Ferguson, 2008) were examined. Correlations among measures were assessed by developmental level and content area (elementary mathematics N = 70; elementary English language arts N = 101; middle school mathematics N = 291, middle school English language arts N = 280). Both average scores and score variability (i.e., coefficient of variation) for the CLASS, FFT, and Tripod were used to predict value-added models (VAM), a high-stakes measure of students' academic growth. For elementary mathematics and ELA, findings indicated the CLASS and FFT exhibited moderate convergent validity while divergent validity was found between the Tripod and the CLASS and FFT. Across content areas in middle school grades, the CLASS, FFT, and Tripod exhibited moderate to high-moderate convergent validity. Average student and observer scores were positively related to VAM scores, whereas variability in scores demonstrated negative relations to VAM scores. Implications of findings for teacher evaluation and professional development are discussed.

Asunto(s)

RESUMEN

CONSTRUCT: We assessed the validity of the modified System for Evaluation of Teaching Qualities (mSETQ) in evaluating clinical teachers in Bahrain. BACKGROUND: Clinical teacher assessment tools are essential for improving teaching quality. The mSETQ is a teaching quality measurement tool, and demonstrating the validity of this tool could provide a stronger evidence base for the utilization of this questionnaire for assessing medical teachers in Bahrain. APPROACH: This study assessed the construct validity of this questionnaire in medical schools across Bahrain using 400 medical students and 149 clinical teachers. Data were analyzed using confirmatory factor analysis (CFA). The goodness-of-fit index (GFI), comparative fit index (CFI), root mean square residual, and standardized root mean square error of approximation (RMSEA) indices were used to evaluate the model fit. The internal consistency reliability was assessed using Cronbach's alpha. RESULTS: The results of the CFA revealed an acceptable fit. All criteria for a good model fit were met except for the RMSEA fit index and the standardized root mean square residual (SRMR) value, which was very close to an acceptable value. Good overall reliability was found in the study (α=0.94). CONCLUSION: The overall findings of this study provided some evidence supporting the reliability and validity of the mSETQ instrument.

RESUMEN

Previous research includes frequent admonitions regarding the importance of establishing connectivity in data collection designs prior to the application of Rasch models. However, details regarding the influence of characteristics of the linking sets used to establish connections among facets, such as locations on the latent variable, model-data fit, and sample size, have not been thoroughly explored. These considerations are particularly important in assessment systems that involve large proportions of missing data (i.e., sparse designs) and are associated with high-stakes decisions, such as teacher evaluations based on teaching observations. The purpose of this study is to explore the influence of characteristics of linking sets in sparsely connected rating designs on examinee, rater, and task estimates. A simulation design whose characteristics were intended to reflect practical large-scale assessment networks with sparse connections were used to consider the influence of locations on the latent variable, model-data fit, and sample size within linking sets on the stability and model-data fit of estimates. Results suggested that parameter estimates for examinee and task facets are quite robust to modifications in the size, model-data fit, and latent-variable location of the link. Parameter estimates for the rater, while still quite robust, are more sensitive to reductions in link size. The implications are discussed as they relate to research, theory, and practice.

RESUMEN

AIM: To assess the trend of changes in the evaluation scores of faculty members and discrepancy between administrators' and students' perspectives in a medical school from 2006 to 2015. MATERIALS AND METHODS: This repeated cross-sectional study was conducted on the 10-year evaluation scores of all faculty members of a medical school (n=579) in an urban area of Iran. Data on evaluation scores given by students and administrators and the total of these scores were evaluated. Data were analyzed using descriptive and inferential statistics including linear mixed effect models for repeated measures via the SPSS software. RESULTS: There were statistically significant differences between the students' and administrators' perspectives over time (p<0.001). The mean of the total evaluation scores also showed a statistically significant change over time (p<0.001). Furthermore, the mean of changes over time in the total evaluation score between different departments was statistically significant (p<0.001). CONCLUSION: The trend of changes in the student's evaluations was clear and positive, but the trend of administrators' evaluation was unclear. Since the evaluation of faculty members is affected by many other factors, there is a need for more future studies.

RESUMEN

OBJECTIVE: To evaluate orthopedic resident perceptions of a didactic curriculum presented in traditional and case-based formats. DESIGN: Prospective cohort study using anonymous web-based survey after each conference evaluating resident perceptions of faculty participation, didactic delivery, content, and overall conference value. Conferences were structured as primarily case-based or traditional lecture. Logistic analysis was performed to determine factors predictive of rating a conference as valuable time spent. SETTING: Orthopedic residency training program at single institution over an academic year. PARTICIPANTS: Orthopedic residents in postgraduate training year 1 to 5 attending mandatory didactic conference. RESULTS: Cased-based conferences received higher Likert ratings on residents' perception of faculty participation, instructor delivery, and improvement in topic understanding when compared to traditional lecture-based conferences (p < 0.0001 for each factor). Residents also were more likely to rate case-based conferences as valuable time spent (p < 0.0001). In our logistic model, factors associated with a negative likelihood of rating a conference as valuable were lecture format (odds ratio [OR] = 0.155, 95% CI: 0.115-0.208), PGY-2 level presenter (OR = 0.288, 95% CI: 0.169-0.490), and PGY-3 level presenter (OR = 0.433, 95% CI: 0.269-0.696). Timing in the year, surgical subspeciality, and conference identity were not significant predictors of conference value rating. CONCLUSIONS: Longitudinal resident feedback demonstrates highly favorable resident perceptions toward case-based formats in didactic sessions. Junior levels residents are not perceived as effective as senior residents and faculty in presenting material in either format. These methods allow for a dynamic approach to identifying strengths and weaknesses in a resident curriculum as a well as a means for more focused and real-time improvements.

Asunto(s)

RESUMEN

PURPOSE: New teacher evaluation systems have expanded the role of principals as instructional leaders, but little is known about principals' ability to promote teacher development through the evaluation process. We conducted a case study of principals' perspectives on evaluation and their experiences implementing observation and feedback cycles to better understand whether principals feel as though they are able to promote teacher development as evaluators. RESEARCH METHODS: We conducted interviews with a stratified random sample of 24 principals in an urban district that recently implemented major reforms to its teacher evaluation system. We analyzed these interviews by drafting thematic summaries, coding interview transcripts, creating data-analytic matrices, and writing analytic memos. FINDINGS: We found that the evaluation reforms provided a common framework and language that helped facilitate principals' feedback conversations with teachers. However, we also found that tasking principals with primary responsibility for conducting evaluations resulted in a variety of unintended consequences which undercut the quality of evaluation feedback they provided. We analyze five broad solutions to these challenges: strategically targeting evaluations, reducing operational responsibilities, providing principal training, hiring instructional coaches, and developing peer evaluation systems. IMPLICATIONS: The quality of feedback teachers receive through the evaluation process depends critically on the time and training evaluators have to provide individualized and actionable feedback. Districts that task principals with primary responsibility for conducting observation and feedback cycles must attend to the many implementation challenges associated with this approach in order for next-generation evaluation systems to successfully promote teacher development.

RESUMEN

Las buenas prácticas docentes requieren del diseño de distintas formas de medición al ser un tema de especial relevancia para determinar la calidad de la educación superior. Una gran parte de las buenas prácticas corresponden a comportamientos desarrollados por el profesor en el aula de clase, lo que se puede medir y analizar mediante metodología observacional. Se seleccionaron diez profesores de la Universidad de La Laguna (España) y siete de la Universidad de Guadalajara (México), codificando sus comportamientos mediante una escala de calificación y los patrones comportamentales de interacción con el alumnado mediante el Protocolo de Observación de Funciones Docentes (PROFUNDO_UNI). El análisis secuencial de retardos extrajo patrones significativos relacionados con la retroalimentación a las preguntas del alumnado, el refuerzo dado y el fomento a la participación de los estudiantes.

The best teaching practices, an issue of particular relevance in determining the quality of higher education, requires different forms of measuring design. A big part of good practice in teacher performance corresponds to behaviors developed by the teacher in the classroom, which can be measured and analyzed using observational methodology. The study evaluates ten teachers from the University of La Laguna (Spain) and seven from Guadalajara (Mexico). Their behavior was coded using a rating scale and the behavioral interaction patterns with students were coded using the Observational Protocol of Teaching Functions (Protocolo de Observación de Funciones Docentes, PROFUNDO_UNI). The lag sequential analysis extracted significant patterns related to the feedback given to questions from the students, the reinforcement and the encouragement given to the students for their participation.

Asunto(s)