基于 VR 技术的沉浸式听障儿童发音康复训练系统

吉林大学学报(信息科学版) ›› 2021, Vol. 39 ›› Issue (6): 700-705.

基于 VR 技术的沉浸式听障儿童发音康复训练系统

杨永吉^a,b , 赵剑^a,b , 史丽娟^a,b , 崔宇晴^b,c , 张智威^c , 肖治国^a,b

长春大学 a. 计算机科学技术学院; b. 人体健康状态辨识与机能增强吉林省重点实验室; c. 电子信息工程学院, 长春 130022

收稿日期:2021-08-21 出版日期:2021-12-01 发布日期:2021-12-02
通讯作者: 赵剑(1980— ), 男, 吉林省吉林市人, 长春大学教授, 博士, 主要从事网络空间安全、虚拟现实与人机交互研究, (Tel)86-431-85251756(E-mail)zhaojian@ ccu. edu. cn。
作者简介:杨永吉(1972— ), 男, 吉林德惠人, 长春大学高级实验师, 主要从事虚拟现实与人机交互研究, (Tel)86-431-85250335(E-mail)yangyj@ ccu. edu. cn.
基金资助:
国家自然科学基金青年基金资助项目(61705251 ); 吉林省科技厅基金资助项目(20200301054RQ; 20200601004JC; 20190201195JC); 吉林省教育厅基金资助项目(JJKH20191207KJ)

Training System of Immersive Pronunciation Rehabilitation for Hearing Impaired Children Based on Virtual Reality VR Technology

YANG Yongji ^a,b , ZHAO Jian ^a,b , SHI Lijuan ^b,c , CUI Yuqing ^b,c , ZHANG Zhiwei ^c , Xiao Zhiguo ^a,b

a. College of Computer Science and Technology; b. Jilin Key Laboratory of Human Health State Identification and Function Enhancement; c. College of Electronic Information Engineering, Changchun University, Changchun 130022, China

Received:2021-08-21 Online:2021-12-01 Published:2021-12-02

摘要/Abstract

摘要： 针对传统听障儿童发音康复训练中患者主动训练积极性逐渐减弱的问题, 设计了一种基于虚拟现实 VR 技术的沉浸式听障儿童发音康复训练系统。系统采用 Unity3D 引擎开发康复场景, 并采用虚拟现实头盔作为输出端展示虚拟环境, 其能以极高的还原度重现虚拟环境, 具有很高的沉浸感。通过面部捕捉技术, 训练出标准的发音、口型等, 通过高清晰的视觉虚拟现实成像系统及手部动作捕捉系统实现人机交互的方式, 触发训练功能, 利用虚拟现实技术实现更好地帮助听障儿童进行沉浸式发音康复训练。实验结果表明, 系统中的发音训练方式可以有效地提高听障儿童的发音能力, 对患者语言能力的康复具有积极作用。

关键词: 虚拟现实技术 , 沉浸式发音康复训练 , Unity3D引擎 , 人机交互

Abstract: Aiming at the problem that patients' enthusiasm for active training is gradually weakened in the traditional pronunciation rehabilitation training of hearing-impaired children, an immersive pronunciation rehabilitation training system for hearing-impaired children based on Virtual Reality VR technology is designed. The system uses the Unity3D Engine to develop the rehabilitation scene and uses the virtual reality helmet as the output to display the virtual environment. It can reproduce the virtual environment with a high degree of restoration and has a high sense of immersion. Through the face capture technology, the standard pronunciation and mouth shape are trained. Through the high-definition visual virtual reality imaging system and hand motion capture system, the human-computer interaction is realized to trigger the training function, and the virtual reality technology is used to better help the hearing-impaired children with immersive pronunciation rehabilitation training. The results show that the pronunciation training method in the system can effectively improve the pronunciation ability of hearing-impaired children and play a positive role in the rehabilitation of patients' language ability.

Key words: virtual reality technology, immersive pronunciation rehabilitation training, Unity3D engine, human-computer interaction

中图分类号:

杨永吉, 赵剑, 史丽娟, 崔宇晴, 张智威, 肖治国. 基于 VR 技术的沉浸式听障儿童发音康复训练系统[J]. 吉林大学学报(信息科学版), 2021, 39(6): 700-705.

YANG Yongji , ZHAO Jian , SHI Lijuan , CUI Yuqing , ZHANG Zhiwei , Xiao Zhiguo. Training System of Immersive Pronunciation Rehabilitation for Hearing Impaired Children Based on Virtual Reality VR Technology[J]. Journal of Jilin University (Information Science Edition), 2021, 39(6): 700-705.

[1]	千承辉, 张昕昊, 陶锦, 刘俊麟. 基于Kinect 的康复训练系统设计与研究[J]. 吉林大学学报(信息科学版), 2020, 38(1): 92-98.
[2]	苏维娜, 刘伟, 张赛鹏. 高校智能电表的设计[J]. 吉林大学学报(信息科学版), 2013, 31(4): 385-388.