SwapSense: Reusable Vision-Based Contact Force Sensing Across Swappable Passive Haptic Modules

Abstract

The haptic qualities of physical interfaces—such as key-click sensations and stylus-tip stiffness—strongly shape user experience beyond functional performance. Prior work has explored reconfigurable passive structures to diversify such haptic experiences, but existing solutions typically sacrifice rich sensing capabilities or require dedicated sensing modules for each configuration. In this paper, we present SwapSense, a modular vision-based haptic I/O framework that physically decouples a shared sensing module from interchangeable haptic augmentation modules. A camera captures sequential deformation of a shared sensing substrate, and an encoder–decoder network estimates contact forces, while each attached augmentation module independently modulates force–displacement characteristics. Our results show that a model trained for pressing force estimation can generalize across users and support effective adaptation to previously unseen augmentation components. Owing to its modular AI architecture, the model efficiently adapts to unseen components with approximately 15 minutes of fine-tuning. A complementary perceptual study further shows that the five haptic augmentations span a perceptually diverse twodimensional perceptual space. Together, these results demonstrate that SwapSense enables reusable contact force sensing and perceptually expressive passive haptic reconfiguration within a single framework.

Minha Jeon

Alumni

Yeseo Park

Undergraduate Researcher

Inhyuk Song

Master’s Student

Seungjae Oh

Assistant Professor

My research interests include interactive I/O technologies, Haptics, and game interaction techniques.