Journal of Jilin University(Engineering and Technology Edition) ›› 2025, Vol. 55 ›› Issue (7): 2402-2408.doi: 10.13229/j.cnki.jdxbgxb.20240598

Previous Articles    

Real-time acquisition and dynamic analysis of learning state based on hybrid intelligence

Hong XIAO(),Xian-de LIU   

  1. School of Computer and Information Technology,Northeast Petroleum University,Daqing 163318,China
  • Received:2024-06-16 Online:2025-07-01 Published:2025-09-12

Abstract:

In the classroom, student state changes are subtle and key features are difficult to capture. Therefore, a real-time collection and dynamic analysis method based on hybrid intelligence is proposed to detect abnormal learning states in a timely manner. By using image enhancement technology to preprocess and collect images, the enhanced images are input into a convolutional neural network. After convolutional processing, deep level feature maps and key feature maps are extracted, and further input into a long short-term memory network to achieve dynamic analysis and recognition of the learning status of classroom students. Through experimental verification, this method can provide real-time feedback on the learning status of students, with good recognition effect and high stability. With the help of this method, students' academic performance can be effectively improved based on the correction of relevant academic personnel, and it has certain application value.

Key words: hybrid intelligence, real-time acquisition of learning status, dynamic analysis, image enhancement technology, convolutional neural network, long short-term memory network

CLC Number: 

  • TP301

Fig.1

Maximum pooling implementation process"

Fig.2

Diagram of long short-term memorynetwork structure"

Fig.3

Learning state acquisition and dynamic analysis structure"

Table 1

Parameters"

算法参数类别参数设置
卷积神经网络网络层数5层(包括1个输入层、3个卷积层、1个全连接层)
卷积层参数卷积核大小:3×3
卷积核数量:第一层64个,后续层翻倍
激活函数:ReLU
最大池化:2×2步长2
全连接层参数神经元数量:512
激活函数:Softmax
优化器Adam
学习率0.001
批大小32
迭代次数100个epoch
损失函数交叉熵损失
长短期记忆网络网络层数3层(包括1个输入层、1个LSTM层、1个全连接层)
LSTM层参数隐藏单元数量:128
激活函数:Sigmoid
全连接层参数神经元数量12
激活函数:Softmax
优化器Adam
学习率0.001
批大小32
迭代次数100个epoch
损失函数交叉熵损失
正则化Dropout率:0.5(在LSTM层后)

Fig.4

Experimental classroom learning status identification page"

Table 2

Results of learning state recognition by proposed method"

编号本文方法识别结果实际情况识别是否一致
1正常正常
2注意力分散注意力分散
3身体不适身体不适
4睡觉睡觉
5注意力分散注意力分散
6睡觉睡觉
7注意力分散注意力分散
8睡觉睡觉
9交头接耳交头接耳
10正常正常
11注意力分散注意力分散
12睡觉睡觉
13睡觉睡觉
14注意力分散注意力分散
15正常正常

Fig.5

Comparison of learning state recognition stability"

Fig.6

Academic record comparison before and after using proposed method"

[1] 姜雨辰, 何赫, 刘涛, 等. 基于人工智能的图书馆学生学习状态监测技术研究[J]. 电子器件, 2023, 46(4): 1070-1074.
Jiang Yu-chen, He He, Liu Tao, et al. Research on student learning state monitoring technology for library based on artificial intelligence[J]. Electronic Devices, 2023, 46(4): 1070-1074.
[2] 林铭炜, 许江松, 林佳胤, 等.面向在线教育的学习者情感识别综述[J]. 控制与决策, 2024, 39(4): 1057-1074.
Lin Ming-wei, Xu Jiang-song, Lin Jia-yin, et al. A review of emotion recognition of learners for online education[J]. Control and Decision, 2024, 39(4): 1057-1074.
[3] 王涛春, 邱庆, 王成田, 等. LD-identify:基于无源RFID的网络学习状态识别[J]. 控制与决策, 2024, 39(1): 219-226.
Wang Tao-chun, Qiu Qing, Wang Cheng-tian, et al. LD-identify:network learning state recognition based on passive RFID[J]. Control and Decision,2024, 39(1): 219-226.
[4] 苏悦, 杨春金, 王建霞. 基于表情识别技术的学生课堂状态检测[J]. 河北工业科技, 2022, 39(5): 396-403.
Su Yue, Yang Chun-jin, Wang Jian-xia. Detection of students' classroom states based on expression recognition technology[J]. Hebei Industry Science and Technology,2022,39(5):396-403.
[5] Chen Z, Liang M, Xue Z,et al.STRAN: student expression recognition based on spatio-temporal residual attention network in classroom teaching videos[J].Applied Intelligence: The International Journal of Artificial Intelligence, Neural Networks, and Complex Problem-Solving Technologies,2023,53(21):25310-25329.
[6] 刘劲, 罗晓曙, 徐照兴. 权重推断与标签平滑的轻量级人脸表情识别[J]. 计算机工程与应用, 2024, 60(2): 254-263.
Liu Jin, Luo Xiao-shu, Xu Zhao-xing. Weight inference and label smoothing for lightweight facial expression recognition[J]. Computer Engineering and Applications, 2024, 60(2): 254-263.
[7] 顾文娟, 丁灿, 魏金, 等.基于双边滤波MSR与AutoMSRCR融合的低光照图像增强[J]. 光学精密工程, 2023, 31(24): 3606-3617.
Gu Wen-juan, Ding Can, Wei Jin, et al. Low-light image enhancement based on the fusion of bilateral filter MSR and AutoMSRCR[J]. Optics and Precision Engineering, 2023, 31(24): 3606-3617.
[8] 李国民, 邵姮, 朱代先, 等. 改进同态滤波与多尺度融合的腐蚀图像增强[J]. 电子测量技术, 2023, 46(13): 118-123.
Li Guo-min, Shao Heng, Zhu Dai-xian, et al. Improved homomorphic filtering and multiscale fusion for corroded image enhancement[J]. Electronic Measurement Technology, 2023, 46(13): 118-123.
[9] 刘波, 田广粮, 肖斌, 等. 利用自适应光照初始化的弱光图像增强方法[J]. 电子与信息学报, 2024, 46(2): 643-651.
Liu Bo, Tian Guang-liang, Xiao Bin, et al. Low light image enhancement with adaptive light initialization[J]. Journal of Electronics & Information Technology, 2024, 46(2): 643-651.
[10] 杜芳芳, 王福忠, 高继梅. 空洞卷积网络下微表情实时识别方法仿真[J]. 计算机仿真, 2023, 40(7):172-175, 461.
Du Fang-fang, Wang Fu-zhong, Gao Ji-mei. Simulation of real-time recognition method of microexpression under empty convolution network[J]. Computer Simulation, 2023, 40(7): 172-175, 461.
[11] 云涛, 潘泉, 郝宇航, 等. 基于HRRP时频特征和多尺度非对称卷积神经网络的目标识别算法[J]. 西北工业大学学报, 2023, 41(3): 537-545.
Yun Tao, Pan Quan, Hao Yu-hang, et al. Target recognition algorithm based on HRRP time-spectrogram feature and multi-scale asymmetric convolutional neural network[J]. Journal of Northwestern Polytechnical University, 2023, 41(3): 537-545.
[12] 任艳丽, 余凌赞, 何港, 等. 一种隐私保护的卷积神经网络预测方案[J]. 计算机学报, 2023, 46(8): 1606-1619.
Ren Yan-li, Yu Ling-zan, He Gang, et al. A scheme of privacy-preserving convolutional neural network prediction[J]. Chinese Journal of Computers,2023, 46(8): 1606-1619.
[13] 冒鑫鑫, 吴胜昔, 咸博龙, 等.基于骨架的自适应图卷积和LSTM行为识别[J]. 华东理工大学学报: 自然科学版, 2022, 48(6): 816-825.
Mao Xin-xin, Wu Sheng-xi, Xian Bo-long, et al. Adaptive graph convolution and LSTM action recognition based on skeleton[J]. Journal of East China University of Science and Technology(Natural Science Edition), 2022, 48(6): 816-825.
[14] 柳长源, 李文强, 毕晓君. 基于RCNN-LSTM的脑电情感识别研究[J]. 自动化学报, 2022, 48(3): 917-925.
Liu Chang-yuan, Li Wen-qiang, Bi Xiao-jun. Research on EEG emotion recognition based on RCNN-LSTM[J]. Acta Automatica Sinica, 2022, 48(3):917-925.
[15] 余金锁, 卢先领. 基于分割注意力的特征融合CNN-Bi-LSTM人体行为识别算法[J]. 电子测量与仪器学报, 2022, 36(2): 89-95.
Yu Jin-shuo, Lu Xian-ling. Human action recognition algorithm of feature fusion CNN-Bi-LSTM based on split-attention[J]. Journal of Electronic Measurement and Instrumentation, 2022, 36(2): 89-95.
[1] Rui-shan DU,Zi-shan WANG. Multi perspective facial expression recognition algorithm based on spatiotemporal attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(6): 2097-2102.
[2] Zhi-gang FENG,Shou-qi WANG,Ming-yue YU. Rolling bearing fault diagnosis based on variational mode extraction and lightweight network [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(6): 1883-1891.
[3] Hao WANG,Bin ZHAO,Guo-hua LIU. Temporal and motion enhancement for video action recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(1): 339-346.
[4] Ping YU,Kang ZHAO,Jie CAO. Rolling bearing fault diagnosis based on optimized A-BiLSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2156-2166.
[5] Sheng-jie ZHU,Xuan WANG,Fang XU,Jia-qi PENG,Yuan-chao WANG. Multi-scale normalized detection method for airborne wide-area remote sensing images [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2329-2337.
[6] Nan YANG,Jun XIAO. Energy saving optimization control of urban intelligent transportation under sequential quadratic programming algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2223-2228.
[7] Xiao-hui WEI,Chen-yang WANG,Qi WU,Xin-yang ZHENG,Hong-mei YU,Heng-shan YUE. Systolic array-based CNN accelerator soft error approximate fault tolerance design [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1746-1755.
[8] Ming-hui SUN,Hao XUE,Yu-bo JIN,Wei-dong QU,Gui-he QIN. Video saliency prediction with collective spatio-temporal attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1767-1776.
[9] Liang-liang GUAN,Guo-hong TIAN. Fault detection algorithm for indicator diagram of automobile shock absorber based on dynamic analysis [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(5): 1221-1226.
[10] Chao XIA,Meng-jia WANG,Jian-yue Zhu,Zhi-gang YANG. Reduced-order modelling of a bluff body turbulent wake flow field using hierarchical convolutional neural network autoencoder [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(4): 874-882.
[11] Guo-jun YANG,Ya-hui QI,Xiu-ming SHI. Review of bridge crack detection based on digital image technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 313-332.
[12] Yu LU,Qian CHEN,Hai-bing YIN. Optimization algorithm for video coding based on convolutional neural networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(11): 3296-3301.
[13] Xi-jun ZHANG,Ji-yang SHANG,Guang-jie YU,Jun HAO. Bearing fault diagnosis based on attention for multi-scale convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(10): 3009-3017.
[14] Xiang-jiu CHE,Huan XU,Ming-yang PAN,Quan-le LIU. Two-stage learning algorithm for biomedical named entity recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2380-2387.
[15] Zhen-hai ZHANG,Kun JI,Jian-wu DANG. Crack identification method for bridge based on BCEM model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1418-1426.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!