RESUMEN

BACKGROUND: We report on our methodological experiences during an investigation of how institutional racism functions in healthcare. We found tension between balancing methodological rigor with the unanticipated consequence of interviewer burden. METHODS: Semi-structured interviews were conducted with patients. Interviews were recorded, transcribed verbatim, and qualitatively analyzed using thematic content analysis. Interviewers also participated in weekly debriefing sessions and reported experiences with patients. RESULTS: Interviewers repeatedly experienced negative encounters with white patients during interviews. Themes included privilege to avoid racism, denial of racism, non-verbal discomfort, falsely claiming Native identities, and intimidation. These experiences were most pronounced with Black interviewers. DISCUSSION: Interviewer burden may need to be a consideration taken up in a variety of research contexts.

Asunto(s)

RESUMEN

The social network perspective has great potential for advancing knowledge of social mechanisms in many fields. However, collecting egocentric (i.e., personal) network data is costly and places a heavy burden on respondents. This is especially true of the task used to elicit information on ties between network members (i.e., alter-alter ties or density matrix), which grows exponentially in length as network size increases. While most existing national surveys circumvent this problem by capping the number of network members that can be named, this strategy has major limitations. Here, we apply random sampling of network members to reduce cost, respondent burden, and error in network studies. We examine the effectiveness and reliability of random sampling in simulated and real-world egocentric network data. We find that in estimating sample/population means of network measures, randomly selecting a small number of network members produces only minor errors, regardless of true network size. For studies that use network measures in regressions, randomly selecting the mean number of network members (e.g., randomly selecting 10 alters when mean network size is 10) is enough to recover estimates of network measures that correlate close to 1 with those of the full sample. We conclude with recommendations for best practices that will make this versatile but resource intensive methodology accessible to a wider group of researchers without sacrificing data quality.

RESUMEN

OBJECTIVE: To describe the adaptive measurement of change (AMC) as a means to identify psychometrically significant change in reported function of hospitalized patients and to reduce respondent burden on follow-up assessments. DESIGN: The AMC method uses multivariate computerized adaptive testing (CAT) and psychometric hypothesis tests based in item response theory to more efficiently measure intra-individual change using the responses of a single patient over 2 or more testing occasions. Illustrations of the utility of AMC in clinical care and estimates of AMC-based item reduction are provided using the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT), a newly developed functional multidimensional CAT-based measurement of basic mobility, daily activities, and applied cognition. SETTING: Two quaternary hospitals in the Upper Midwest. PARTICIPANTS: Four hundred ninety-five hospitalized patients who completed the FAMCAT on 2 to 4 occasions during their hospital stay. INTERVENTION: N/A. RESULTS: Of the 495 patients who completed more than 1 FAMCAT, 72% completed 2 sessions, 13% completed 3, and 15% completed 4, with 22.1%, 23.4%, and 23.0%, respectively, exhibiting significant multivariate change. Use of the AMC in conjunction with the FAMCAT reduced respondent burden from that of the FAMCAT alone for follow-up assessments. On average, when used without the AMC, 22.7 items (range, 20.4-24.4) were administered during FAMCAT sessions. Post hoc analyses determined that when the AMC was used with the FAMCAT a mean±standard deviation reduction in FAMCAT number of items of 13.6 (11.1), 13.1 (9.8), and 18.1 (10.8) would occur during the second, third, and fourth sessions, respectively, which corresponded to a reduction in test duration of 3.0 (2.4), 3.0 (2.8), and 4.7 (2.6) minutes. Analysis showed that the AMC requires no assumptions about the nature of change and provides data that are potentially actionable for patient care. Various patterns of significant univariate and multivariate change are illustrated. CONCLUSIONS: The AMC method is an effective and parsimonious approach to identifying significant change in patients' measured CAT scores. The AMC approach reduced FAMCAT sessions by an average of 12.6 items (55%) and 2.9 minutes (53%) among patients with psychometrically significant score changes.

Asunto(s)

RESUMEN

Large-scale, nationally representative surveys serve many vital functions, but these surveys are often long and burdensome for respondents. Cutting survey length can help to reduce respondent burden and may improve data quality but removing items from these surveys is not a trivial matter. We propose a method to empirically assess item importance and associated burden in national surveys and guide this decision-making process using different research products produced from such surveys. This method is demonstrated using the Survey of Doctorate Recipients (SDR), a biennial survey administered to individuals with a Science, Engineering, and Health doctorate. We used three main sources of information on the SDR variables: 1) a bibliography of documents using the SDR data, 2) the SDR website that allows users to download summary data, and 3) web timing paradata and break-off rates. The bibliography was coded for SDR variable usage and citation counts. Putting this information together, we identified 35 unused items (17% of the survey) by any of these sources and found that the most burdensome items are highly important. We conclude with general recommendations for those hoping to employ similar methodologies in the future.

RESUMEN

Respondent burden has important implications for survey outcomes, including response rates and attrition in panel surveys. Despite this, respondent burden remains an understudied topic in the field of survey methodology, with few researchers systematically measuring objective and subjective burden factors in surveys used to produce official statistics. This research was designed to assess the impact of proxy measures of respondent burden, drawing on both objective (survey length and frequency), and subjective (effort, saliency, and sensitivity) burden measures on response rates over time in the Current Population Survey (CPS). Exploratory Factor Analysis confirmed the burden proxy measures were interrelated and formed five distinct factors. Regression tree models further indicated that both objective and subjective proxy burden factors were predictive of future CPS response rates. Additionally, respondent characteristics, including employment and marital status, interacted with these burden factors to further help predict response rates over time. We discuss the implications of these findings, including the importance of measuring both objective and subjective burden factors in production surveys. Our findings support a growing body of research suggesting that subjective burden and individual respondent characteristics should be incorporated into conceptual definitions of respondent burden and have implications for adaptive design.

RESUMEN

BACKGROUND: The Screener and Opioid Assessment for Patients with Pain-Revised (SOAPP-R) is a 24-item self-report instrument that was developed to aid providers in predicting aberrant medication-related behaviors among chronic pain patients. Although the SOAPP-R has garnered widespread use, certain patients may be dissuaded from taking it because of its length. Administrative barriers associated with lengthy questionnaires further limit its utility. OBJECTIVE: To investigate the extent to which two techniques for computer-based administration (curtailment and stochastic curtailment) reduce the average test length of the SOAPP-R without unduly affecting sensitivity and specificity. DESIGN: Retrospective study. SETTING: Pain management centers. SUBJECTS: Four hundred and twenty-eight chronic non-cancer pain patients. METHODS: Subjects had taken the full-length SOAPP-R and been classified by the Aberrant Drug Behavior Index (ADBI) as having engaged or not engaged in aberrant medication-related behavior. Curtailment and stochastic curtailment were applied to the data in post-hoc simulation. Sensitivity and specificity with respect to the ADBI, as well as average test length, were computed for the full-length test, curtailment, and stochastic curtailment. RESULTS: The full-length SOAPP-R exhibited a sensitivity of 0.745 and a specificity of 0.671 for predicting the ADBI. Curtailment reduced the average test length by 26% while exhibiting the same sensitivity and specificity as the full-length test. Stochastic curtailment reduced the average test length by as much as 65% while always exhibiting sensitivity and specificity for the ADBI within 0.035 of those of the full-length test. CONCLUSIONS: Curtailment and stochastic curtailment have potential to improve the SOAPP-R's efficiency in computer-based administrations.

Asunto(s)

RESUMEN

OBJECTIVE: In primary care, screening instruments for mental health should be ultrashort to allow for routine usage. In this paper, Stochastic Curtailment is introduced as a method for constructing ultrashort screeners. METHOD: In a post hoc diagnostic accuracy study using the item scores on the Patient Health Questionnaire depression module (PHQ-9) of a large sample (N=20, 685), Stochastic Curtailment was compared with two existing ultrashort versions of PHQ-9. The first was PHQ-2 (which includes the first two items of PHQ-9), and the second was a 'two-step' method (only if the PHQ-2 screened positive were all nine PHQ items administered). For PHQ-2 and two-step, both cut scores 2 and 3 were evaluated. RESULTS: PHQ-2 showed the lowest and Stochastic Curtailment the highest diagnostic accuracy with reference to the classifications based on the full PHQ-9. To do so, Stochastic Curtailment used 3.08 items on average (S.D.=1.98), which was slightly less than two-step (M=3.18, S.D.=2.62) under its most accurate cut score (≥2). CONCLUSIONS: It was concluded that Stochastic Curtailment is a promising method for constructing ultrashort screeners.

Asunto(s)

RESUMEN

Health questionnaires are often built up from sets of questions that are totaled to obtain a sum score. An important consideration in designing questionnaires is to minimize respondent burden. An increasingly popular method for efficient measurement is computerized adaptive testing; unfortunately, many health questionnaires do not meet the requirements for this method. In this paper, a new sequential method for efficiently obtaining sum scores via the computer is introduced, which does not have such requirements and is based on the ordinal regression model. In the assessment, future scores are predicted from past responses, and when an acceptable level of uncertainty is achieved, the procedure is terminated. Two simulation studies were performed to illustrate the usefulness of the procedure. The first used artificially generated symptom scores, and the second was a post hoc simulation using real responses on the Center for Epidemiologic Studies Depression scale. In both studies, the sequential method substantially reduced the respondent burden while maintaining a high sum score quality. Benefits and limitations of this new methodology are discussed.

Asunto(s)

RESUMEN

Minimizing the respondent burden and maximizing the classification accuracy of tests is essential for efficacious screening for common mental health disorders. In previous studies, curtailment of tests has been shown to reduce average test length considerably, without loss of accuracy. In the current study, we simulate Deterministic (DC) and Stochastic (SC) Curtailment for three self-report questionnaires for common mental health disorders, to study the potential gains in efficiency that can be obtained in screening for these disorders. The curtailment algorithms were applied in an existing dataset of item scores of 502 help-seeking participants. Results indicate that DC reduces test length by up to 37%, and SC reduces test length by up to 46%, with only very slight decreases in diagnostic accuracy. Compared to an item response theory based adaptive test with similar test length, SC provided better diagnostic accuracy. Consequently, curtailment may be useful in improving the efficiency of mental health self-report questionnaires.

Asunto(s)

RESUMEN

Survey respondents may misinterpret the questions they are asked, potentially undermining the accuracy of their answers. One way to reduce this risk is to make definitions of key question concepts available to the respondents. In the current study we compared two methods of making definitions available to web survey respondents - displaying the definition with the question text and displaying the definition when respondents roll the mouse over the relevant question terms. When definitions were always displayed they were consulted more than when they required a rollover request. The length of the definitions did not affect how frequently they were used under either method of display. Respondents who completed training items designed to encourage definition use actually requested definitions less often, suggesting that they may value minimal effort over improved understanding. We conclude that at least for small numbers of questions, providing definitions with the question is likely to be the more effective approach than rollovers or hyperlinks.