OBJECTIVES: To develop and implement a pediatric clinical skills assessment (PCSA) for residents, using children as standardized patients (SPs); to assess the psychometric adequacy of the PCSA and use it to evaluate the performance of residents; and to evaluate the feasibility of using child SPs and the response of the residents and the child SPs to participation in the PCSA. METHODS: Ten 22-minute complete patient encounters were developed, 7 with child SPs. Fifty-six residents (10 second-year pediatric residents, 29 first-year pediatric residents, and 17 first-year family practice residents) were evaluated on the following clinical skills: history taking, physical examination, interpersonal skills, and documentation and interpretation of clinical data/patient note. MAIN OUTCOME MEASURES: Patient encounter checklists, focus groups, and questionnaires. RESULTS: Average skill scores for the 56 residents were 68% (SD, 12%) for history taking, 56% (SD, 26%) for physical examination, 46% (SD, 12%) for patient note, and 68% (SD, 16%) for interpersonal skills. Second-year pediatric residents scored significantly higher on history taking than first-year pediatric and first-year family practice residents; first-year pediatric residents scored significantly higher on interpersonal skills than second-year pediatric and first-year family practice residents; and first- and second-year pediatric residents scored significantly higher on the patient note component than first-year family practice residents. All differences noted were significant at P<.05. There were no significant differences on physical examination between the groups. Reliabilities were 0.69 for history taking, 0.64 for physical examination, 0.76 for interpersonal skills, and 0.81 for the patient note component. On a Likert scale (5 indicates high; 1, low), residents rated the PCSA 3.9 for realism, 4.1 for challenge, 3.1 for enjoyment, and 2.9 for fairness. Child SPs found the experience positive. No negative effects on the children were identified by their real parents or their SP parents. CONCLUSIONS: Our development method gives content validity to our PCSA, and resident scores give indication of PCSA construct validity. Reliabilities are in the acceptable range. Residents found the PCSA challenging and realistic but less than enjoyable and fair. Use of child SPs is feasible. Resident performance scores were low relative to the performance criteria of the PCSA development group. The adequacy of clinical skills teaching and assessment in residency programs needs to be reviewed. Deficits in specific skills and overall performance of residents identified by a PCSA could be used to guide individual remediation and curricular change.