Self-Supervised Transfer Learning for IMU-Based Upper-Limb Action Detection and Motion Quality Analysis in an Immersive VR Functional Task

Wearable inertial sensing has considerable potential for process-level analysis of upper-limb function, but further evidence is needed to understand how it can be applied within ecologically structured immersive virtual reality (VR) tasks. Most VR-based functional assessments rely primarily on outcome-level indicators, such as task completion time, success rate, or error count, which may not fully capture how a task is executed. This exploratory study investigated whether wearable IMU signals collected during an immersive VR sushi-making task could support binary detection of a core upper-limb manipulation phase and provide additional information about task execution beyond global performance outcomes. A total of 45 participants contributed usable motion recordings for this study, with five Xsens DOT sensors placed on the hands, forearms, and waist. Three signal modalities were analysed, including acceleration (ACC), gyroscope angular velocity (GYR), and Euler angles. The downstream recognition problem was formulated as a binary classification task (Placing vs. Non-Placing), and a self-supervised learning (SSL) pretrain–fine-tune strategy was evaluated against conventional machine learning and from-scratch deep learning baselines using five subject-wise validation splits. The strongest overall performance was achieved with hand-mounted accelerometer signals, with LeftHand–ACC achieving a Macro-F1 of 0.712±0.128 and RightHand–ACC achieving 0.679±0.118. Under both hand-ACC settings, SSL fine-tuning showed higher mean Macro-F1 than the Balanced Random Forest baseline and the same deep architecture trained from scratch. Recognition performance varied substantially across sensor locations, signal modalities, and task segments, with distal upper-limb sensors generally outperforming waist-based configurations. Cross-age analyses further showed that within-cohort and cross-cohort performance did not fully align, indicating sensitivity to age-related distribution shift. Beyond classification, Log Dimensionless Jerk (LDLJ) derived from the Placing action showed a significant positive association with Cognitron motor control time cost (𝑟=0.636, 𝑝<0.001). These findings suggest that wearable IMU sensing can provide preliminary process-level information during immersive VR functional tasks, including task-phase detection, sensing-configuration comparison, cross-cohort generalisation assessment, and exploratory motion-quality analysis. The results should be interpreted as evidence of feasibility rather than as a mature biomechanical or clinical assessment model.