RESUMEN
This article presents results from an experimental study in Germany designed to test the effectiveness of a novel protocol for matching participants in a national panel survey with interviewers employing computer-assisted telephone interviewing (CATI) on selected sociodemographic features, including sex, age, and education. We specifically focus on the ability of the protocol to engender close matches between respondents and interviewers in terms of these features, using both theory and empirical evidence to suggest that this type of matching will improve cooperation rates in surveys employing CATI. We also focus on indicators of "success" at first contact (defined as a successful interview or establishment of an appointment for an interview) as a function of whether the matching protocol was in use on a given day and whether specific types of matches generated higher rates of success overall. We find strong evidence of the protocol effectively establishing close matches, and we also observe that matches based on education proved especially effective for rates of "success" in a panel survey that focused primarily on labor market topics. We conclude with thoughts on practical implementation of this approach in other settings and suggested directions for future work in this area.
RESUMEN
Using list-assisted random digit dialing (RDD) with telephone data collection and address-based sampling (ABS) with mail questionnaires are two survey designs that yield probability based inference, yet they are so different that they can yield entirely different results. The 2007 Health Information National Trends Survey (HINTS) provides a unique opportunity to evaluate the effect of these different designs on a variety of survey estimates and, even more importantly, the effect on individual sources of survey error. Understanding the difference in error structure between the two designs is important to survey practitioners in order to select the optimum design, and to data users who can anticipate which results may be affected and how. We first compared estimates between the two designs and then estimated the different sources of error. In addition to identified differences in estimates, we found that for some estimates the two designs can yield similar results merely due to the effect of similar biases. The error components were quite different between the two designs--while the ABS design yields almost complete coverage of the population compared to the RDD design, it was subjected to substantially higher nonresponse bias.
Asunto(s)
Encuestas Epidemiológicas/métodos , Servicios Postales/métodos , Proyectos de Investigación , Teléfono , Sesgo , Humanos , MuestreoRESUMEN
A near ubiquitous feature of user interfaces is feedback on task completion or progress indicators such as the graphical bar that grows as more of the task is completed. The presumed benefit is that users will be more likely to complete the task if they see they are making progress but it is also possible that feedback indicating slow progress may sometimes discourage users from completing the task. This paper describes two experiments that evaluate the impact of progress indicators on the completion of on-line questionnaires. In the first experiment, progress was displayed at different speeds throughout the questionnaire. If the early feedback indicated slow progress, abandonment rates were higher and users' subjective experience more negative than if the early feedback indicated faster progress. In the second experiment, intermittent feedback seemed to minimize the costs of discouraging feedback while preserving the benefits of encouraging feedback. Overall, the results suggest that when progress seems to outpace users' expectations, feedback can improve their experience though not necessarily their completion rates; when progress seems to lag behind what users expect, feedback degrades their experience and lowers completion rates.
RESUMEN
Survey respondents may misinterpret the questions they are asked, potentially undermining the accuracy of their answers. One way to reduce this risk is to make definitions of key question concepts available to the respondents. In the current study we compared two methods of making definitions available to web survey respondents - displaying the definition with the question text and displaying the definition when respondents roll the mouse over the relevant question terms. When definitions were always displayed they were consulted more than when they required a rollover request. The length of the definitions did not affect how frequently they were used under either method of display. Respondents who completed training items designed to encourage definition use actually requested definitions less often, suggesting that they may value minimal effort over improved understanding. We conclude that at least for small numbers of questions, providing definitions with the question is likely to be the more effective approach than rollovers or hyperlinks.
RESUMEN
BACKGROUND: Attrition, or dropout, is a problem faced by many online health interventions, potentially threatening the inferential value of online randomized controlled trials. OBJECTIVE: In the context of a randomized controlled trial of an online weight management intervention, where 85% of the baseline participants were lost to follow-up at the 12-month measurement, the objective was to examine the effect of nonresponse on key outcomes and explore ways to reduce attrition in follow-up surveys. METHODS: A sample of 700 non-respondents to the 12-month online follow-up survey was randomly assigned to a mail or telephone nonresponse follow-up survey. We examined response rates in the two groups, costs of follow-up, reasons for nonresponse, and mode effects. We ran several logistic regression models, predicting response or nonresponse to the 12-month online survey as well as predicting response or nonresponse to the follow-up survey. RESULTS: We analyzed 210 follow-up respondents in the mail and 170 in the telephone group. Response rates of 59% and 55% were obtained for the telephone and mail nonresponse follow-up surveys, respectively. A total of 197 respondents (51.8%) gave reasons related to technical issues or email as a means of communication, with older people more likely to give technical reasons for non-completion; 144 (37.9%) gave reasons related to the intervention or the survey itself. Mail follow-up was substantially cheaper: We estimate that the telephone survey cost about US $34 per sampled case, compared to US $15 for the mail survey. The telephone responses were subject to possible social desirability effects, with the telephone respondents reporting significantly greater weight loss than the mail respondents. The respondents to the nonresponse follow-up did not differ significantly from the 12-month online respondents on key outcome variables. CONCLUSIONS: Mail is an effective way to reduce attrition to online surveys, while telephone follow-up might lead to overestimating the weight loss for both the treatment and control groups. Nonresponse bias does not appear to be a significant factor in the conclusions drawn from the randomized controlled trial.