RESUMEN

Seeing the visual articulatory movements of a speaker, while hearing their voice, helps with understanding what is said. This multisensory enhancement is particularly evident in noisy listening conditions. Multisensory enhancement also occurs even in auditory-only conditions: auditory-only speech and voice-identity recognition is superior for speakers previously learned with their face, compared to control learning; an effect termed the "face-benefit". Whether the face-benefit can assist in maintaining robust perception in increasingly noisy listening conditions, similar to concurrent multisensory input, is unknown. Here, in two behavioural experiments, we examined this hypothesis. In each experiment, participants learned a series of speakers' voices together with their dynamic face, or control image. Following learning, participants listened to auditory-only sentences spoken by the same speakers and recognised the content of the sentences (speech recognition, Experiment 1) or the voice-identity of the speaker (Experiment 2) in increasing levels of auditory noise. For speech recognition, we observed that 14/30 participants (47%) showed a face-benefit. While 19/25 participants (76%) showed a face-benefit for voice-identity recognition. For those participants who demonstrated a face-benefit, the face-benefit increased with auditory noise levels. Taken together, the results support an audio-visual model of auditory communication and suggest that the brain can develop a flexible system in which learned facial characteristics are used to deal with varying auditory uncertainty.

RESUMEN

According to semiotics, we live in a world of signs, where almost anything can act as a signifier and convey meaning. But what of the semiotic landscape of midwifery? What signs are present within a client's multi-sensory experience of their midwifery care? How are these signs functioning to increase equity and accessibility? Or worse, how might certain aspects of the client's experience communicate unjust power dynamics? Semiotics allows us to examine a wide communicative and educational environment. By paying particular attention to the multivalent meanings of different signs-be they written, visual, oral, or even physical-we can start to see how multimodal communication plays a vital role in a client's perception of equity and power. One way to improve client experience is by approaching education and semiotic experience from the same place as trauma-informed care. A more health-literate sensitive approach viewed through the lens of semiotics assumes all clients have little previous knowledge or comfort within a care setting. This hyperawareness and criticality of the semiotic environment would allow midwives to acknowledge various sensory and communicative biases and intentionally redesign the entire client experience. The semiotic landscape is then curated to meet the needs of the most important audience-those marginalized and discriminated against whether that is because of education, finances, race, gender, or any other intersectional identity. We must acknowledge the fact that all sign systems can either reinforce abusive power relations or work to improve them. For what is at stake here is not just a client's overall comfort, but their full understanding of the care they are receiving, the options they have, and their autonomy within their entire perinatal experience.

RESUMEN

Digital technology creates new opportunities to design multisensory learning experiences. Evidence suggests that digital innovation can greatly benefit health education, including nutrition programs. The COVID-19 pandemic disrupted the education sector, forcing schools to modify standard practices from exclusively in-person delivery to online or blended learning. Digitalized curriculums became particularly useful as an Emergency Remote Teaching tool. This article focuses on developing and implementing a multimedia, multisensory, and scalable Hip-Hop Healthy Eating and Living in Schools (H.E.A.L.S.) Nutrition-Math Curriculum (NMC). NMC comprises 20 lessons-music-based multimedia resources used in the classroom or at home. Fourteen lessons represent self-directed online modules (asynchronous learning) hosted on a Learning Management System (LMS) called "Gooru." The remaining six lessons are teacher-facilitated (in person or using Zoom) review sessions (synchronous learning). The article discusses (1) the development of NMC through the lens of the Multisensory Multilevel Health Education Model (MMHEM), (2) the high acceptability of NMC evaluated using a mixed-methods design among minoritized fifth-grade students attending an after-school program, and (3) the students' completion and mastery rates of the NMC modules based on LMS data. Multimedia nutrition education programs integrated with common core curriculum content, such as NMC, may be a promising avenue for disseminating health education to minoritized children living in New York City and similar high fast-food density cities.

RESUMEN

In the field of learning theory and practice, the superior efficacy of multisensory learning over uni-sensory is well-accepted. However, the underlying neural mechanisms at the macro-level of the human brain remain largely unexplored. This study addresses this gap by providing novel empirical evidence and a theoretical framework for understanding the superiority of multisensory learning. Through a cognitive, behavioral, and electroencephalographic assessment of carefully controlled uni-sensory and multisensory training interventions, our study uncovers a fundamental distinction in their neuroplastic patterns. A multilayered network analysis of pre- and post- training EEG data allowed us to model connectivity within and across different frequency bands at the cortical level. Pre-training EEG analysis unveils a complex network of distributed sources communicating through cross-frequency coupling, while comparison of pre- and post-training EEG data demonstrates significant differences in the reorganizational patterns of uni-sensory and multisensory learning. Uni-sensory training primarily modifies cross-frequency coupling between lower and higher frequencies, whereas multisensory training induces changes within the beta band in a more focused network, implying the development of a unified representation of audiovisual stimuli. In combination with behavioural and cognitive findings this suggests that, multisensory learning benefits from an automatic top-down transfer of training, while uni-sensory training relies mainly on limited bottom-up generalization. Our findings offer a compelling theoretical framework for understanding the advantage of multisensory learning.

Asunto(s)

RESUMEN

Prior research investigating whether and how multisensory information facilitates skill learning is quite mixed; whereas some research points to congruent information improving learning, other work suggests that people become reliant on the redundant information, such that its removal ultimately detracts from the ability to perform a unisensory task. We examined this question using the Serial Interception Sequence Learning (SISL) task, a visuo-motor paradigm in which participants implicitly learn a sequence embedded in noise. We investigated whether adding auditory information in different ways would enhance real time sequence learning and whether any benefits of multisensory learning would persist with visual-only testing. Auditory information was used either as feedback on the visuo-motor task (Experiments 1 and 2) or was presented synchronously with visual information during learning (Experiment 3). Robust sequence-specific performance advantages occurred across conditions and experiments; however, auditory information enhanced real-time performance only when it was synchronized with visual information. Participants were significantly more accurate, faster, and more precise with stimulus-locked auditory information during training. Notably, these benefits did not generalize to the visual-only context, suggesting that the benefits of stimulus-locked auditory information are primarily useful only when the perceptual information is present.

RESUMEN

Purpose: The purpose of this paper is to derive into practical recommendations from multisensory stimulation with virtual reality (VR) and scent to help educators develop effective teaching strategies geared toward aspects of the learning experience, recall, and creativity in a stereotypical learning context. Design/methodology/approach: The paper is based on a randomized experiment in which student participants were subdivided into three treatment groups and one control group. Each group was stimulated by a different combination of visual, auditory, and olfactory stimuli (2D SMELL, VR, and VR SMELL) and the outcomes were compared against those of the control group (2D). Consistent with the Cognitive Theory of Multimedia Learning, hypotheses were constructed to study the effect of different combinations of stimuli on the learning experience and learning outcomes related to recall and creativity in a stereotypical learning context. Findings: Traditional video content alone and bundled with a coherent olfactory stimulus prompted higher self-reported ratings of perceived quality of the sensory experience. Olfactory stimulus in combination with either VR or a traditional video prompted higher self-reported ratings on perceived immersion. In a stereotypical learning context, the highest recall scores were achieved with traditional video alone. Both VR alone and bundled with an olfactory stimulus resulted in enhanced creativity. Research limitations/implications: The findings of this study should be interpreted in the context of adopting multisensory stimulations combined with VR technology as part of stereotypical learning contexts. Most professional educators do not have robust knowledge or experience in using build-on-purpose multisensory stimuli but are increasingly engaged in using multisensory tools such as VR, as part of their teaching practice. In relation to recall, the results are consistent with the hypothesis that in a stereotypical learning context, a multisensory experience involving VR and olfactory stimuli can be related to an undesired cognitive load for learners. There exists a possibility that the low-technical version of the VR goggles used, as well as the contents of the instructional video may have influenced the learning outcomes in terms of recall. Hence, future research should consider such aspects and focus on richer learning contexts. Originality/value: This work offers practical recommendations for instructional design strategies aiming to create multisensory stimulations with VR and olfactory components to foster a richer learning experience and enhanced learning outcomes, under the assumptions of a stereotypical learning context.

RESUMEN

Most studies of memory and perceptual learning in humans have employed unisensory settings to simplify the study paradigm. However, in daily life we are often surrounded by complex and cluttered scenes made up of many objects and sources of sensory stimulation. Our experiences are, therefore, highly multisensory both when passively observing the world and when acting and navigating. We argue that human learning and memory systems are evolved to operate under these multisensory and dynamic conditions. The nervous system exploits the rich array of sensory inputs in this process, is sensitive to the relationship between the sensory inputs, and continuously updates sensory representations, and encodes memory traces based on the relationship between the senses. We review some recent findings that demonstrate a range of human learning and memory phenomena in which the interactions between visual and auditory modalities play an important role, and suggest possible neural mechanisms that can underlie some surprising recent findings. We outline open questions as well as directions of future research to unravel human perceptual learning and memory.

RESUMEN

Integration of sensory signals that emanate from the same source, such as the visual of lip articulations and the sound of the voice of a speaking individual, can improve perception of the source signal (e.g., speech). Because momentary sensory inputs are typically corrupted with internal and external noise, there is almost always a discrepancy between the inputs, facing the perceptual system with the problem of determining whether the two signals were caused by the same source or different sources. Thus, whether or not multisensory stimuli are integrated and the degree to which they are bound is influenced by factors such as the prior expectation of a common source. We refer to this factor as the tendency to bind stimuli, or for short, binding tendency. In theory, the tendency to bind sensory stimuli can be learned by experience through the acquisition of the probabilities of the co-occurrence of the stimuli. It can also be influenced by cognitive knowledge of the environment. The binding tendency varies across individuals and can also vary within an individual over time. Here, we review the studies that have investigated the plasticity of binding tendency. We discuss the protocols that have been reported to produce changes in binding tendency, the candidate learning mechanisms involved in this process, the possible neural correlates of binding tendency, and outstanding questions pertaining to binding tendency and its plasticity. We conclude by proposing directions for future research and argue that understanding mechanisms and recipes for increasing binding tendency can have important clinical and translational applications for populations or individuals with a deficiency in multisensory integration.

RESUMEN

The conventional physiology courses consist of theoretical lectures, clinical application seminars, numerical exercises, simulations, and laboratory practices. However, in subjects that involve relevant physical quantities, even students who successfully pass exams may be unable to realize the actual quantities involved. For example, students may know what the values of the aortic diameter and cardiac output are, and they may be skilled at calculating changes in variables without being able to realize the actual physical magnitudes of the variables, resulting in limited understanding. To address this problem, here we describe and discuss simple practical exercises specifically designed to allow students to multisensory experience (touch, see, hear) the actual physical magnitudes of aortic diameter and cardiac output in adult humans at rest and exercise. The results obtained and the feedback from a student survey both clearly show that the described approach is a simple and interesting tool for motivating students and providing them with more realistic learning.

Asunto(s)

RESUMEN

Multisensory learning profits from stimulus congruency at different levels of processing. In the current study, we sought to investigate whether multisensory learning can potentially be based on high-level feature congruency (same meaning) without perceptual congruency (same time) and how this relates to changes in brain function and behaviour. 50 subjects learned to decode Morse code (MC) either in unisensory or different multisensory manners. During unisensory learning, the MC was trained as sequences of auditory trains. For low-level congruent (perceptual) multisensory learning, MC was applied as tactile stimulation to the left hand simultaneously to the auditory stimulation. In contrast, high-level congruent multisensory learning involved auditory training, followed by the production of MC sequences requiring motor actions and thereby excludes perceptual congruency. After learning, group differences were observed within three distinct brain regions while processing unisensory (auditory) MC. Both types of multisensory learning were associated with increased activation in the right inferior frontal gyrus. Multisensory low-level learning elicited additional activation in the somatosensory cortex, while multisensory high-level learners showed a reduced activation in the inferior parietal lobule, which is relevant for decoding MC. Furthermore, differences in brain function associated with multisensory learning was related to behavioural reaction times for both multisensory learning groups. Overall, our data support the idea that multisensory learning is potentially based on high-level features without perceptual congruency. Furthermore, learning of multisensory associations involves neural representations of stimulus features involved in learning, but also share common brain activation (i.e. the right IFG), which seems to serve as a site of multisensory integration.

Asunto(s)

RESUMEN

Adherence to antiretroviral therapy (ART) among youth remains low. We piloted an adapted active visualization device that demonstrates how ART works in the body. Youth living with HIV were randomized to: (1) standard care (n = 14) or the (2) adapted active visualization intervention (n = 14) and 71% of the sample (n = 19) were re-assessed on viral load, adherence behaviors, and illness perceptions 2.5 months later. Intervention youth had lower viral loads, reported less difficulty in adhering to ART, and more motivation and control over their HIV than standard care at follow-up. Active visualization may be an acceptable tool to address ART adherence among youth.

Asunto(s)

RESUMEN

The present study used magnetoencephalography (MEG) to identify the neural correlates of audiovisual statistical learning, while disentangling the differential contributions of uni- and multi-modal statistical mismatch responses in humans. The applied paradigm was based on a combination of a statistical learning paradigm and a multisensory oddball one, combining an audiovisual, an auditory and a visual stimulation stream, along with the corresponding deviances. Plasticity effects due to musical expertise were investigated by comparing the behavioral and MEG responses of musicians to non-musicians. The behavioral results indicated that the learning was successful for both musicians and non-musicians. The unimodal MEG responses are consistent with previous studies, revealing the contribution of Heschl's gyrus for the identification of auditory statistical mismatches and the contribution of medial temporal and visual association areas for the visual modality. The cortical network underlying audiovisual statistical learning was found to be partly common and partly distinct from the corresponding unimodal networks, comprising right temporal and left inferior frontal sources. Musicians showed enhanced activation in superior temporal and superior frontal gyrus. Connectivity and information processing flow amongst the sources comprising the cortical network of audiovisual statistical learning, as estimated by transfer entropy, was reorganized in musicians, indicating enhanced top-down processing. This neuroplastic effect showed a cross-modal stability between the auditory and audiovisual modalities.

Asunto(s)

RESUMEN

Temporal and spatial characteristics of sensory inputs are fundamental to multisensory integration because they provide probabilistic information as to whether or not multiple sensory inputs belong to the same event. The multisensory temporal binding window defines the time range within which two stimuli of different sensory modalities are merged into one percept and has been shown to depend on training. The aim of the present study was to evaluate the role of the training procedure for improving multisensory temporal discrimination and to test for a possible transfer of training to other multisensory tasks. Participants were trained over five sessions in a two-alternative forced-choice simultaneity judgment task. The task difficulty of each trial was either at each participant's threshold (adaptive group) or randomly chosen (control group). A possible transfer of improved multisensory temporal discrimination on multisensory binding was tested with a redundant signal paradigm in which the temporal alignment of auditory and visual stimuli was systematically varied. Moreover, the size of the spatial audio-visual ventriloquist effect was assessed. Adaptive training resulted in faster improvements compared to the control condition. Transfer effects were found for both tasks: The processing speed of auditory inputs and the size of the ventriloquist effect increased in the adaptive group following the training. We suggest that the relative precision of the temporal and spatial features of a cross-modal stimulus is weighted during multisensory integration. Thus, changes in the precision of temporal processing are expected to enhance the likelihood of multisensory integration for temporally aligned cross-modal stimuli.

RESUMEN

Individuals vary in their tendency to bind signals from multiple senses. For the same set of sights and sounds, one individual may frequently integrate multisensory signals and experience a unified percept, whereas another individual may rarely bind them and often experience two distinct sensations. Thus, while this binding/integration tendency is specific to each individual, it is not clear how plastic this tendency is in adulthood, and how sensory experiences may cause it to change. Here, we conducted an exploratory investigation which provides evidence that (1) the brain's tendency to bind in spatial perception is plastic, (2) that it can change following brief exposure to simple audiovisual stimuli, and (3) that exposure to temporally synchronous, spatially discrepant stimuli provides the most effective method to modify it. These results can inform current theories about how the brain updates its internal model of the surrounding sensory world, as well as future investigations seeking to increase integration tendencies.

RESUMEN

Fundamental concepts in biochemistry important for drug design often lack connection to the macroscopic world and can be difficult for students to grasp, particularly those in introductory science courses at the high school and college level. Educational research has shown that multisensory teaching facilitates learning, but teaching at the high school and college level is almost exclusively limited to the visual and auditory senses. This approach neglects the lifetime of experience our students bring to the classroom in the form of taste perception and makes our teaching less supportive of those with sensory impairment. In this article, we outline a novel guided-inquiry activity that utilizes taste perception for a series of natural and artificial sweetener solutions to introduce the concepts of substrate affinity and selectivity in the context of drug design. The findings from this study demonstrate clear gains in student knowledge, as well as an increase in enthusiasm for the fields of biochemistry and drug design. © 2016 by The International Union of Biochemistry and Molecular Biology, 44(6):550-554, 2016.

Asunto(s)

RESUMEN

Training studies, in which the structural or functional neurophysiology is compared before and after expertise is acquired, are increasingly being used as models for understanding the human brain's potential for reorganization. It is proving difficult to use these results to answer basic and important questions like how task training leads to both specific and general changes in behavior and how these changes correspond with modifications in the brain. The main culprit is the diversity of paradigms used as complex task models. An assortment of activities ranging from juggling to deciphering Morse code has been reported. Even when working in the same general domain, few researchers use similar training models. New ways to meaningfully compare complex tasks are needed. We propose a method for characterizing and deconstructing the task requirements of complex training paradigms, which is suitable for application to both structural and functional neuroimaging studies. We believe this approach will aid brain plasticity research by making it easier to compare training paradigms, identify "missing puzzle pieces," and encourage researchers to design training protocols to bridge these gaps.

RESUMEN

Multisensory perception has been the focus of intense investigation in recent years. It is now well-established that crossmodal interactions are ubiquitous in perceptual processing and endow the system with improved precision, accuracy, processing speed, etc. While these findings have shed much light on principles and mechanisms of perception, ultimately it is not very surprising that multiple sources of information provides benefits in performance compared to a single source of information. Here, we argue that the more surprising recent findings are those showing that multisensory experience also influences the subsequent unisensory processing. For example, exposure to auditory-visual stimuli can change the way that auditory or visual stimuli are processed subsequently even in isolation. We review three sets of findings that represent three different types of learning ranging from perceptual learning, to sensory recalibration, to associative learning. In all these cases exposure to multisensory stimuli profoundly influences the subsequent unisensory processing. This diversity of phenomena may suggest that continuous modification of unisensory representations by multisensory relationships may be a general learning strategy employed by the brain.