RESUMEN
Objective monitoring and assessment of human motor behavior can improve the diagnosis and management of several medical conditions. Over the past decade, significant advances have been made in the use of wearable technology for continuously monitoring human motor behavior in free-living conditions. However, wearable technology remains ill-suited for applications which require monitoring and interpretation of complex motor behaviors (e.g., involving interactions with the environment). Recent advances in computer vision and deep learning have opened up new possibilities for extracting information from video recordings. In this paper, we present a hierarchical vision-based behavior phenotyping method for classification of basic human actions in video recordings performed using a single RGB camera. Our method addresses challenges associated with tracking multiple human actors and classification of actions in videos recorded in changing environments with different fields of view. We implement a cascaded pose tracker that uses temporal relationships between detections for short-term tracking and appearance based tracklet fusion for long-term tracking. Furthermore, for action classification, we use pose evolution maps derived from the cascaded pose tracker as low-dimensional and interpretable representations of the movement sequences for training a convolutional neural network. The cascaded pose tracker achieves an average accuracy of 88% in tracking the target human actor in our video recordings, and overall system achieves average test accuracy of 84% for target-specific action classification in untrimmed video recordings.
Asunto(s)
Monitoreo Fisiológico , Actividad Motora/fisiología , Grabación en Video/métodos , Algoritmos , Humanos , Procesamiento de Imagen Asistido por Computador , Redes Neurales de la ComputaciónRESUMEN
Prairie voles are socially monogamous rodents that form social bonds similar to those seen in primates. Social behavior investigation in these species, that include studying their breathing regulation, can provide us with an invaluable psychological model to understand social and emotional functions in both animals and humans. There have been several studies associated with the respiratory pattern of these species in the state of fear-induced defense. However, non-invasive measurement methods employed so far suffer from the lack of a natural experiment environment for the rodents. In this paper, we present a remote depth-based system, which applies a modified autocorrelation algorithm to automatically extract respiration patterns in small rodents. We evaluated our estimation accuracy through a series of experiments and comparing the extracted results with breathing rates obtained from visual inspection of synchronously collected RGB videos. In a preliminary test on a human participant, breathing rate was estimated with 100% accuracy, while the estimation accuracy was 94.8% for a restrained vole. Finally, we monitored the respiratory alternations of three voles in transition from a baseline, to a fearful state, and back to a normal state; the estimated breathing rates confirmed the existing hypothesis regarding animal defense strategies.