基于伪触觉反馈的多模态车载交互

doi:10.13229/j.cnki.jdxbgxb.20250497

摘要/Abstract

摘要：

为提升车载交互系统在复杂驾驶场景中的反馈效率与感知一致性，构建了融合视觉、听觉与触觉的伪触觉信号系统，并验证了其在典型工况下的实用性。系统基于电容屏与压电执行器设计振动参数矩阵，建立感知属性与振动参数的映射关系，采用克里金插值构建三维感知曲面。提出了频率映射与谐波包络两种信号生成方法，后者在频率一致性与信号区分度方面表现更优。辨识试验设置4个刺激等级，视觉模态识别性能最优，触觉模态对刺激数量变化更敏感。在3.2 km驾驶仿真场景中，A+H（+V）共发掩蔽次任务配置在多数工况中横摆角RMS和峰值最低，表明该次任务条件下驾驶稳定性最好，且操作时间与主观评分方面综合表现最优。

关键词: 计算机应用, 伪触觉, 多模态反馈, 车载交互, 谐波包络, 掩蔽技术

Abstract:

This study aims to enhance the feedback efficiency and perceptual consistency of in-vehicle interaction systems under complex driving conditions by developing a pseudo-haptic signal system that integrates visual， auditory， and tactile modalities. The system is built using a capacitive touchscreen and piezoelectric actuators， with a designed vibration parameter matrix to model the mapping between perceptual attributes and vibration features. Kriging interpolation is employed to construct three-dimensional perceptual surfaces. Two signal generation methods—frequency mapping and harmonic envelope modulation—are proposed， with the latter showing better performance in frequency alignment and signal discriminability. A recognition experiment was conducted using four levels of stimulus quantity. Results indicate that the visual modality achieved the highest identification performance， while the tactile modality was more sensitive to changes in stimulus number. A 3.2 km driving simulation scenario was constructed to represent typical driving conditions. Under this setup， the auditory-tactile concurrent masking configuration（A+H（+V）） yielded the lowest RMS and peak yaw rates across most conditions， indicating the highest driving stability under this secondary task， along with the best overall performance in terms of response time and subjective evaluation.

Key words: computer application, pseudo-haptics, multimodal feedback, in-vehicle interaction, harmonic envelope modulation, masking technology

中图分类号:

U461

沈传亮,叶瑞章,马骁远,张嘉垚,汪震,胡宏宇. 基于伪触觉反馈的多模态车载交互[J]. 吉林大学学报(工学版), 2026, 56(10): 2524-2534.

Chuan-liang SHEN,Rui-zhang YE,Xiao-yuan MA,Jia-yao ZHANG,Zhen WANG,Hong-yu HU. Pseudo⁃haptic feedback for multimodal in⁃vehicle interaction[J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(10): 2524-2534.

图/表 16

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

图11

图12

图13

图14

图15

参考文献 27

[1]	Gao F, Ge X, Li J, et al. Intelligent cockpits for connected vehicles: taxonomy, architecture, interaction technologies, and future directions[J]. Sensors, 2024, 24(16): 5172.
[2]	Kun A L, Schmidt A, Dey A, et al. Automotive user interfaces and interactive applications in the car[J]. Personal and Ubiquitous Computing, 2013, 17(5): 801-802.
[3]	Prabhakar G, Biswas P. A brief survey on interactive automotive UI[J]. Transportation Engineering, 2021, 6: 100089.
[4]	Erolin J. Haptic Feedback Defined: Types, Benefits & Uses[EB/OL]. [2026-04-13]. .
[5]	Ng A, Brewster S A. Investigating pressure input and haptic feedback for In-car touchscreens and touch surfaces[C]∥Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, New York, USA: 2016: 121-128.
[6]	马晓娜, 童倩倩, 李一含, 等. 基于多模态触觉与伪触觉反馈的智能设计触觉体验构建研究[J]. 装饰, 2022(9): 28-33.
	Ma Xiao-na, Tong Qian-qian, Li Yi-han, et al. Research on the construction of haptic experience in intelligent design based on multimodal haptics and pseudo-haptic feedback[J]. Art & Design, 2022(9): 28-33.
[7]	Lecuyer A, Coquillart S, Kheddar A, et al. Pseudo-haptic feedback: can isometric input devices simulate force feedback?[C]∥Proceedings IEEE Virtual Reality 2000(Cat. No.00CB37048), New Brunswick, USA, 2000: 83-90.
[8]	Hachisu T, Cirio G, Marchal M, et al. Pseudo-haptic feedback augmented with visual and tactile vibrations[C]∥IEEE International Symposium on VR Innovation, Singapore, 2011: 327-328.
[9]	Magnusson C, Rassmus-Gröhn K. A pilot study on audio induced pseudo-haptics[C]∥Proceedings of the 3rd International Haptic and Auditory Interaction Design Workshop, Jyväskylä, Finland, 2008: 6-7.
[10]	De-Pra Y, Catrambone V, van-Wassenhove V, et al. Altering time perception in virtual reality through multimodal visual-tactile kappa effect[J]. IEEE Transactions on Haptics, 2023, 16(4): 518-523.
[11]	Jiang P, Kent C, Rossiter J. Towards sensory substitution and augmentation: mapping visual distance to audio and tactile frequency[J]. PLOS One, 2024, 19(3): e0299213.
[12]	Zhu A, Choi A T H, Ma K H, et al. Tactile vibrating toolkit and driving simulation platform for driving-related research[J]. Journal of Visualized Experiments: JoVE, 2020(166): e61408.
[13]	Miyoshi J K, Chen Q, Li J, et al. Effects of instructional vibrotactile patterns on takeover performance in autonomous driving[C]∥International Conference on Consumer Electronics, Taiwan, China, 2023: 377-378.
[14]	Gibson J J. Observations on active touch[J]. Psychological Review, 1962, 69(6): 477-491.
[15]	Vega-Bermudez F, Johnson K O, Hsiao S S. Human tactile pattern recognition: Active versus passive touch, velocity effects, and patterns of confusion[J]. Journal of Neurophysiology, 1991, 65(3): 531-546.
[16]	Smith A M, Chapman C E, Donati F, et al. Perception of simulated local shapes using active and passive touch[J]. Journal of Neurophysiology, 2009, 102(6): 3519-3529.
[17]	Ackerley R, Hassan E, Curran A, et al. An fMRI study on cortical responses during active self-touch and passive touch from others[J]. Frontiers in Behavioral Neuroscience, 2012, 6: 51.
[18]	Cybulska-Klosowicz A, Tremblay F, Jiang W, et al. Differential effects of the mode of touch, active and passive, on experience-driven plasticity in the S1 cutaneous digit representation of adult macaque monkeys[J]. Journal of Neurophysiology, 2020, 123(3): 1072-1089.
[19]	He B, Bian N, Wu Y, et al. Multimodal interaction strategies for take-over requests in L3 automated driving[C]∥The 9th International Symposium on Sensors, Mechatronics, and Automation System(ISSMAS 2023), Nanjing, China,2023: 421-427.
[20]	Ahmaniemi T, Marila J, Lantz V. Design of dynamic vibrotactile textures[J]. IEEE Transactions on Haptics, 2010, 3(4): 245-256.
[21]	Strohmeier P, Hornbæk K. Generating haptic textures with a vibrotactile actuator[C]∥Proceedings of the CHI Conference on Human Factors in Computing Systems, Denver, USA, 2017: 4994-5005.
[22]	Wu S, Sun X, Wang Q, et al. Tactile modeling and rendering image-textures based on electrovibration[J]. The Visual Computer, 2016, 33(5): 637-646.
[23]	Wang Y, Akeju O V, Zhao T. Interpolation of spatially varying but sparsely measured geo-data: a comparative study[J]. Engineering Geology, 2017, 231: 200-217.
[24]	Duc M L, Bilik P, Martinek R. Harmonics signal feature extraction techniques: a review[J]. Mathematics, 2023, 11(8): 1877.
[25]	Zhang J. Music feature extraction and classification algorithm based on deep learning[J]. Scientific Programming, 2021, 2021(1): 1651560.
[26]	Ludeña-Choez J, Gallardo-Antolín A. Feature extraction based on the high-pass filtering of audio signals for acoustic event classification[J]. Computer Speech & Language, 2015, 30(1): 32-42.
[27]	Bennasar M, Hicks Y, Setchi R. Feature selection using joint mutual information maximisation[J]. Expert Systems with Applications, 2015, 42(22): 8520-8532.

相关文章 15

[1]	王兴旺,罗锦宽,张紫洋,迟玉璞,蒋金佑,于美铭. 基于改进型DQN的无人机覆盖路径规划算法[J]. 吉林大学学报(工学版), 2026, 56(9): 2444-2455.
[2]	卢家伟,剡昌锋. 基于真⁃伪三连体神经网络的传感器本体匹配方法[J]. 吉林大学学报(工学版), 2026, 56(9): 2456-2466.
[3]	艾青林,刘元宵,杨佳豪. 基于MFF-STDC网络的室外复杂环境小目标语义分割方法[J]. 吉林大学学报(工学版), 2025, 55(8): 2681-2692.
[4]	张宇飞,王丽敏,赵建平,贾智尧,李明洋. 基于中心选择大逃杀优化算法的机器人逆运动学求解[J]. 吉林大学学报(工学版), 2025, 55(8): 2703-2710.
[5]	朴燕,康继元. RAUGAN：基于循环生成对抗网络的红外图像彩色化方法[J]. 吉林大学学报(工学版), 2025, 55(8): 2722-2731.
[6]	车翔玖,李良. 融合全局与局部细粒度特征的图相似度度量算法[J]. 吉林大学学报(工学版), 2025, 55(7): 2365-2371.
[7]	李文辉,杨晨. 基于对比学习文本感知的小样本遥感图像分类[J]. 吉林大学学报(工学版), 2025, 55(7): 2393-2401.
[8]	庄珊娜,王君帅,白晶,杜京瑾,王正友. 基于三维卷积与自注意力机制的视频行人重识别[J]. 吉林大学学报(工学版), 2025, 55(7): 2409-2417.
[9]	刘琼昕,王甜甜,王亚男. 非支配排序粒子群遗传算法解决车辆位置路由问题[J]. 吉林大学学报(工学版), 2025, 55(7): 2464-2474.
[10]	王健,贾晨威. 面向智能网联车辆的轨迹预测模型[J]. 吉林大学学报(工学版), 2025, 55(6): 1963-1972.
[11]	车翔玖,孙雨鹏. 基于相似度随机游走聚合的图节点分类算法[J]. 吉林大学学报(工学版), 2025, 55(6): 2069-2075.
[12]	赵宏伟,周伟民. 基于数据增强的半监督单目深度估计框架[J]. 吉林大学学报(工学版), 2025, 55(6): 2082-2088.
[13]	周丰丰,郭喆,范雨思. 面向不平衡多组学癌症数据的特征表征算法[J]. 吉林大学学报(工学版), 2025, 55(6): 2089-2096.
[14]	刘萍萍,商文理,解小宇,杨晓康. 基于细粒度分析的不均衡图像分类算法[J]. 吉林大学学报(工学版), 2025, 55(6): 2122-2130.
[15]	陈海鹏,张世博,吕颖达. 多尺度感知与边界引导的图像篡改检测方法[J]. 吉林大学学报(工学版), 2025, 55(6): 2114-2121.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

编号	实验条件	简写（V/A/H）
1	视	V
2	听（+视）	A（+V）
3	触（+视）	H（+V）
4	听+触（+视）共发掩蔽	A+H（+V）共发掩蔽
5	听+触（+视）后向掩蔽	A+H（+V）后向掩蔽
6	听+触（+视）基座掩蔽	A+H（+V）基座掩蔽
7	仅驾驶任务无次任务	仅驾驶任务