RESUMEN

The development and dissemination of meaningful and useful performance reports associated with examinations involved in the licensure process are important to the communities of interest, including state boards, candidates, and professional schools. Discussions of performance reporting have been largely neglected however. The authors recognize and reinforce the need for such discussions by providing prototypes of performance reporting in dentistry with examples and recommendations to guide practice. For illustrative purposes, this article reviews and discusses the different reporting models used over the past ten years with Part I and Part II of the National Board Dental Examination (NBDE). These reporting models are distinguished by such features as the following: 1) scores in each discipline covered on the exam (four for Part I and nine for Part II) and an overall average are reported in a standard-score metric; 2) a single overall score in a standard-score metric is reported; and 3) performance on the exam is reported as pass/fail. Standard scores on the NBDE range from 49 to 99, with 75 being a passing score. Sample data, without identifying information, are used to illustrate the reporting models.

Asunto(s)

RESUMEN

This article describes a protocol that was employed to validate the content of the National Board Dental Hygiene Examination (NBDHE) using two separate practice analyses. The protocol consisted of the following phases: (1) the careful definition of the domain of knowledge and skills required for successful entry-level dental hygiene practice, (2) the conduct of a survey to gather information regarding the judgments of practicing dental hygienists on the important knowledge, abilities, and skills required of entry-level dental hygiene practice, (3) the analysis of the survey data, (4) the integration of the survey findings with the examination content using a two-dimensional matrix, (5) development of the updated specifications of the examination based on the practice analysis data, and (6) implementation of the updated specifications for subsequent item and test development. Survey data from 1,284 and 1,388 full-time practicing dental hygienists for the 2002 and 2009 practice analyses, respectively, were analyzed. A two-dimensional matrix was used to map the content of the examination to the practice of dental hygiene based on the practice analysis data. The mapping showed that the distribution of items in the content specifications reflected actual dental hygiene practice. Thus, the adequacy of the content validity of the NBDHE was confirmed.

Asunto(s)

RESUMEN

The Standards for Educational and Psychological Testing emphasize the importance of documenting and describing the procedures followed in developing valid test content. As a result, the Joint Commission on National Dental Examinations, the testing agency responsible for administering the National Board Dental Examination Part I and Part II, routinely communicates information about the validity of Part II to dental schools and other communities of interest. Since 2000, the content of Part II has been validated through the use of three practice analyses. This article provides an overview of these practice analyses, including procedures and findings. In general, the findings confirm that the content of Part II is valid in determining the qualifications of individuals seeking dental licensure.

Asunto(s)

RESUMEN

In support of actions taken by the Joint Commission on National Dental Examinations, two changes--the inclusion of testlet items and the random presentation of items in an interdisciplinary format--were made to enhance the test validity of the National Board Dental Examination Part I in 2007. As a result, the examination was changed from a conjunctive to a comprehensive format. It was assumed that validity would be enhanced with regard to the examination's internal structure, while not disturbing item performance and examinee score. This study of the results found that 1) three underlying variables were extracted from the conjunctive Part I but only two underlying variables from the comprehensive Part I and 2) the differences in item performance and examinee score were generally small in effect size across formats. Factor analyses revealed that Part I was more discipline-sensitive for the conjunctive format but more item format-sensitive for the comprehensive format. The revision of Part I changed the nature of the examination from a discipline-based format to a more clinically relevant, interdisciplinary format, a favorable outcome anticipated by the Joint Commission. The results of this study provide evidence supporting the validity of the revised Part I examination for its intended purpose in the licensure process.

Asunto(s)

RESUMEN

The purpose of this study was to explore the patterns that emerge among stress, burnout, and renewal activities of dental hygiene education administrators in six midwestern states in the United States. The study investigated the effects of stress on these administrators by identifying when stress and burnout occur, what precautions they take to prevent it, and what actions might combat stress and/or burnout once it has occurred. The administrators were asked to complete a demographic questionnaire, the Maslach Burnout Inventory (MBI)-Educators Survey, and an in-depth interview. The response rate to the demographic questionnaire and MBI-Educators Survey was 54.5 percent (30/55). Respondents were primarily Caucasian females (93 percent), at least fifty-one years of age (67 percent), employed in dental hygiene education at least twenty-one years (56 percent), and dental hygiene education administrators for less than ten years (55 percent). Almost half (43 percent) reported a moderate to high Emotional Exhaustion burnout score, one of three characteristics measured by the MBI-Educators Survey. All participants (100 percent) responded that stress had affected their personal and/or professional lives. The findings indicate that dental hygiene education administrators a) experience stress, b) experience patterns of stress, and c) use preventive strategies. Study participants felt that the stress and burnout they experienced may be altered through personal and/or professional lifestyle modifications and that additional training in stress management is needed.

Asunto(s)

Asunto(s)

RESUMEN

In this article, the Task Force on Student Outcomes Assessment of the American Dental Education Association's Commission on Change and Innovation in Dental Education describes the current status of student outcomes assessment in U.S. dental education. This review is divided into six sections. The first summarizes the literature on assessment of dental students' performance. Section two discusses catalysts, with a focus on problem-based learning, for development of new assessment methods, while the third section presents several resources and guides that can be used to inform selection of assessment techniques for various domains of competence. The fourth section describes the methodology and results of a 2008 survey of current assessment practices in U.S. dental schools. In the fifth section, findings from this survey are discussed within the context of competency-based education, the educational model for the predoctoral curriculum endorsed by the American Dental Education Association and prescribed by the Commission on Dental Accreditation. The article concludes with a summary of assessments recommended as optimal strategies to measure three components of professional competence based on the triangulation model. The survey of assessment practices in predoctoral education was completed by 931 course directors, representing 45 percent of course directors nationwide, from fifty-three of the fifty-six U.S. dental schools. Survey findings indicate that five traditional mainstays of student performance evaluation-multiple-choice testing, lab practicals, daily grades, clinical competency exams, and procedural requirements-still comprise the primary assessment tools in dental education. The survey revealed that a group of newer assessment techniques, although frequently identified as best practices in the literature and commonly used in other areas of health professions education, are rarely employed in predoctoral dental education.

Asunto(s)

RESUMEN

A 28-item inventory was developed to measure the clinical problem-solving abilities of 3rd and 4th year dental students. The judgments of 57 expert raters (dental-school faculty) from four dental schools used the inventory to evaluate 183 dental students on a 5-point rating scale. The Rasch measurement model was employed to examine the psychometric properties and construct validity of this inventory. In this study, fit statistics identified the "noise" in the data and residual analysis assisted in extracting a meaningful structure. The study results indicate that the Rasch measurement model appeared to be a useful method for use in producing a unidimensional instrument. All five rating categories were used in a coherent manner, and four discernable levels of clinical problem-solving ability were identified. After removal of four repetitious items, a version of the Clinical Problem-Solving Inventory was finalized that could serve as a criterion measure for validating the use of a critical thinking test on the Dental Admission Test.

Asunto(s)

RESUMEN

Subscores resulting from the administration of high-stakes tests to candidates for credentials in the health professions are desirable for two reasons. First, failing candidates want a profile of performance to plan future remedial studies. Second, training institutions want a profile of performance for their graduates to better evaluate their training. The validity of the interpretation or use of subscores depends on a summative judgment based on a combination of reasoning and empirical analyses, known as validation. We describe this reasoning process and show that with a large credentialing test the validity of any subscore interpretation or use can and should be studied systematically. Validity evidence should be established to support the interpretation and use of subscores that we intend to report. Some principles arise in this study related to the validity of subscores, and some procedures are proposed to help testing program personnel better validate the use of subscores.

Asunto(s)

RESUMEN

Successful completion of Part II of the National Board Dental Examinations is a part of the licensure process for dentists. Good testing practice requires that the content of a high stakes examination like Part II be based on a strong relationship between the content and the judgments of practicing dentists on what is important to their practice of dentistry. In an effort to demonstrate this relationship for Part II, the Joint Commission conducted a practice analysis, which involved a two-dimensional model. The sixty-three Competencies of the New Dentist, developed and promulgated by the American Dental Education Association, were used for one dimension, and the current content specifications were used for the other. A survey of 520 practicing dentists was conducted to determine the importance of each of the competencies for patient care. These dentists were recent graduates of accredited programs and passed Part II three to five years prior to the conduct of the practice analysis. The survey directed the respondents to rate the importance of the competencies on a scale from 1 to 5. Of the 520 in the sample, 244 dentists responded. The reliability index was above 0.90. The importance rating for each competency was translated into the associated number of items. The number of items devoted to each competency was allocated to the current content elements that are related to the knowledge and problem-solving skills that support each competency. The findings specified revisions in the relative number of items dedicated to the various elements in the specifications. These findings indicate that the items on the examination under the current distribution adequately reflected practice. In general, there were relatively small changes in the content specifications. The total number of changes in items was forty-eight, which represents changes in slightly less than 10 percent of the overall number of items.

Asunto(s)

RESUMEN

This study examined item calibration stability in relation to response time and the levels of item difficulty between different response time groups on a sample of 389 examinees responding to six different subtest items of the Perceptual Ability Test (PAT). The results indicated that no Differential Item Functioning (DIF) was found and a significant correlation coefficient of item difficulty was formed between slow and fast responders. Three distinct levels of difficulty emerged among the six subtests across groups. Slow responders spent significantly more time than fast responders on the four most difficult subtests. A positive significant relationship was found between item difficulty and response time across groups on the overall perceptual ability test items. Overall, this study found that: 1) the same underlying construct is being measured across groups, 2) the PAT scores were equally useful across groups, 3) different sources of item difficulty may exist among the six subtests, and 4) more difficult test items may require more time to answer.

Asunto(s)