RESUMEN
BACKGROUND: Teleguidance, a promising telemedicine service for intraoperative surgical consultation, was planned to scale up at a major academic hospital in partnership with 5 other hospitals. If the service was adopted and used over time, it was expected to provide educational benefits and improve clinical outcomes during endoscopic retrograde cholangiopancreatography (ERCP), which is a technically advanced procedure for biliary and pancreatic disease. However, it is known that seemingly successful innovations can play out differently in new settings, which might cause variability in clinical outcomes. In addition, few telemedicine services survive long enough to deliver system-level outcomes, the causes of which are not well understood. OBJECTIVE: We were interested in factors related to usability and user experience of the telemedicine service, which might affect adoption. Therefore, we investigated perceptions and responses to the use and anticipated use of a system. Technology acceptance, a construct referring to how users perceive a technology's usefulness, is commonly considered to indicate whether a new technology will actually be used in a real-life setting. Satisfaction measures were used to investigate whether user expectations and needs have been met through the use of technology. In this study, we asked surgeons to rate the perceived usefulness of teleguidance, and their satisfaction with the telemedicine service in direct conjunction with real-time use during clinical procedures. METHODS: We designed domain-specific measures for perceived usefulness and satisfaction, based on performance and outcome measures for the clinical procedure. Surgeons were asked to rate their user experience with the telemedicine service in direct conjunction with real-time use during clinical procedures. RESULTS: In total, 142 remote intraoperative consultations were conducted during ERCP procedures at 5 hospitals. The demand for teleguidance was more pronounced in cases with higher complexity. Operating surgeons rated teleguidance to have contributed to performance and outcomes to a moderate or large extent in 111 of 140 (79.3%) cases. Specific examples were that teleguidance was rated as having contributed to intervention success and avoiding a repeated ERCP in 23 cases, avoiding 3 PTC, and 11 referrals, and in 11 cases, combinations of these outcomes. Preprocedure beliefs about the usefulness of teleguidance were generally lower than postprocedure satisfaction ratings. The usefulness of teleguidance was mainly experienced through practical advice from the consulting specialist (119/140, 85%) and support with assessment and decision-making (122/140, 87%). CONCLUSIONS: Users' satisfaction with teleguidance surpassed their initial expectations, mainly through contribution to nontechnical aspects of performance, and through help with general assessment. Teleguidance shows the potential to improve performance and outcomes during ERCP. However, it takes hands-on experience for practitioners to understand how the new telemedicine service contributes to performance and outcomes.
RESUMEN
Many promising telemedicine innovations fail to be accepted and used over time, and there are longstanding questions about how to best evaluate telemedicine services and other health information technologies. In response to these challenges, there is a growing interest in how to take the sociotechnical complexity of health care into account during design, implementation, and evaluation. This paper discusses the methodological implications of this complexity and how the sociotechnical context holds the key to understanding the effects and outcomes of telemedicine. Examples from a work domain analysis of a surgical setting, where a telemedicine service for remote surgical consultation was to be introduced, are used to show how abstracted functional modeling can provide a structured and rigorous means to analyze and represent the implementation context in complex health care settings.
RESUMEN
BACKGROUND: A telemedicine service enabling remote surgical consultation had shown promising results. When the service was to be scaled up, it was unclear how contextual variations among different clinical sites could affect the clinical outcomes and implementation of the service. It is generally recognized that contextual factors and work system complexities affect the implementation and outcomes of telemedicine. However, it is methodologically challenging to account for context in complex health care settings. We conducted a work domain analysis (WDA), an engineering method for modeling and analyzing complex work environments, to investigate and represent contextual influences when a telemedicine service was to be scaled up to multiple hospitals. OBJECTIVE: We wanted to systematically characterize the implementation contexts at the clinics participating in the scale-up process. Conducting a WDA would allow us to identify, in a systematic manner, the functional constraints that shape clinical work at the implementation sites and set the sites apart. The findings could then be valuable for informed implementation and assessment of the telemedicine service. METHODS: We conducted observations and semistructured interviews with a variety of stakeholders. Thematic analysis was guided by concepts derived from the WDA framework. We identified objects, functions, priorities, and values that shape clinical procedures. An iterative "discovery and modeling" approach allowed us to first focus on one clinic and then readjust the scope as our understanding of the work systems deepened. RESULTS: We characterized three sets of constraints (ie, facets) in the domain: the treatment facet, administrative facet (providing resources for procedures), and development facet (training, quality improvement, and research). The constraints included medical equipment affecting treatment options; administrative processes affecting access to staff and facilities; values and priorities affecting assessments during endoscopic retrograde cholangiopancreatography; and resources for conducting the procedure. CONCLUSIONS: The surgical work system is embedded in multiple sets of constraints that can be modeled as facets of the system. We found variations between the implementation sites that might interact negatively with the telemedicine service. However, there may be enough motivation and resources to overcome these initial disruptions given that values and priorities are shared across the sites. Contrasting the development facets at different sites highlighted the differences in resources for training and research. In some cases, this could indicate a risk that organizational demands for efficiency and effectiveness might be prioritized over the long-term outcomes provided by the telemedicine service, or a reduced willingness or ability to accept a service that is not yet fully developed or adapted. WDA proved effective in representing and analyzing these complex clinical contexts in the face of technological change. The models serve as examples of how to analyze and represent a complex sociotechnical context during telemedicine design, implementation, and assessment.
RESUMEN
BACKGROUND: Telemedicine innovations are rarely adopted into routine health care, the reasons for which are not well understood. Teleguidance, a promising service for remote surgical guidance during endoscopic retrograde cholangiopancreatography (ERCP) was due to be scaled up, but there were concerns that user attitudes might influence adoption. OBJECTIVE: Our objective was to gain a deeper understanding of ERCP practitioners' attitudes toward teleguidance. These findings could inform the implementation process and future evaluations. METHODS: We conducted semistructured interviews with ERCP staff about challenges during work and beliefs about teleguidance. Theoretical constructs from the technology acceptance model (TAM) guided the thematic analysis. Our findings became input to a 16-item questionnaire, investigating surgeons' beliefs about teleguidance's contribution to performance and factors that might interact with implementation. RESULTS: Results from 20 interviews with ERCP staff from 5 hospitals were used to adapt a TAM questionnaire, exchanging the standard "Ease of Use" items for "Compatibility and Implementation Climate." In total, 23 ERCP specialists from 15 ERCP clinics responded to the questionnaire: 9 novices (<500 ERCP procedures) and 14 experts (>500 ERCP procedures). The average agreement ratings for usefulness items were 64% (~9/14) among experts and 75% (~7/9) among novices. The average agreement ratings for compatibility items were somewhat lower (experts 64% [~9/14], novices 69% [~6/9]). The averages have been calculated from the sum of several items and therefore, they only approximate the actual values. While 11 of the 14 experts (79%) and 8 of the 9 novices (89%) agreed that teleguidance could improve overall quality and patient safety during ERCP procedures, only 8 of the 14 experts (57%) and 6 of the 9 novices (67%) agreed that teleguidance would not create new patient safety risks. Only 5 of the 14 experts (36%) and 3 of the 9 novices (33%) were convinced that video and image transmission would function well. Similarly, only 6 of the 14 experts (43%) and 6 of the 9 novices (67%) agreed that administration would work smoothly. There were no statistically significant differences between the experts and novices on any of the 16 items (P<.05). CONCLUSIONS: Both novices and experts in ERCP procedures had concerns that teleguidance might disrupt existing work practices. However, novices were generally more positive toward teleguidance than experts, especially with regard to the possibility of developing technical skills and work practices. While newly trained specialists were the main target for teleguidance, the experts were also intended users. As experts are more likely to be key decision makers, their attitudes may have a greater relative impact on adoption. We present suggestions to address these concerns. We conclude that using the TAM as a conceptual framework can support user-centered inquiry into telemedicine design and implementation by connecting qualitative findings to well-known analytical themes.
RESUMEN
As costs for healthcare are rising in society, information systems are often seen as enablers of new cost-saving healthcare processes. But an on-going deployment of a wide range of new kinds of systems requires close attention to interoperability between new and legacy systems. Another challenge is to assure that the healthcare professions are given realistic opportunities to play an active part in designing the new ways of working that the new, integrated systems are designed to support. We argue that a feasible way to approach such a user participation in design of work processes and systems is to extend well known user-survey and strategy-mapping methods with the new value-based healthcare approach which invites health professionals to participate in strategic assessments of health outcome and costs along the care chain in which they work. We also argue that such a combination of practical research methods resonates well with Techno-Anthropology's foregrounding of ethical considerations to inform the interdisciplinary cross-fertilization of interactional competencies in health informatics research.