论文中文题名: | 基于言语想象的文本生成技术研究与实现 |
姓名: | |
学号: | 21206227088 |
保密级别: | 保密(1年后开放) |
论文语种: | chi |
学科代码: | 085800 |
学科名称: | 工学 - 能源动力 |
学生类型: | 硕士 |
学位级别: | 工学硕士 |
学位年度: | 2024 |
培养单位: | 西安科技大学 |
院系: | |
专业: | |
研究方向: | 脑机接口 |
第一导师姓名: | |
第一导师单位: | |
论文提交日期: | 2024-06-19 |
论文答辩日期: | 2024-06-06 |
论文外文题名: | Research and Implementation of Text Generation TechnologyBased on Speech imagery |
论文中文关键词: | |
论文外文关键词: | Brain-computer interface ; Speech imagery ; Feature extraction ; Classification ; Decode ; Text generation |
论文中文摘要: |
脑机接口(Brain–Computer Interface, BCI)技术可以通过解读大脑中的无声发音神 经信号来重建言语意图,帮助患有严重言语生成障碍的人恢复交流能力。本文通过对 离散采集的字符信号进行可分性验证,并对连续采集的语句信号进行标签标注及解码, 实现在线文本生成及实时校正。主要研究内容如下: 1. 构建了字符级言语想象脑电(Electroencephalogram, EEG)信号数据集,选择 26 个英文字母及5个标点符号作为想象材料。首先,提出小波散射变换(Wavelet Scattering Transform, WST)结合核主成分分析(Kernel Principal Component Analysis, KPCA)的方法提取时频特征。其次,利用极致梯度提升(ExtremeGradientBoosting, XGBoost)分类器对字符信号进行分类,平均准确率为77.89%。最后,通过进一步聚 类分析和对比研究验证了信号可分性。 2. 构建了句子级EEG数据集,通过连续想象进行信号采集,以解决离散信号采集 的局限性。首先,采用动态时间扭曲(DynamicTimeWarping,DTW)和隐马尔科夫模 型(HiddenMarkovModel, HMM)进行时间校准和标签标注。其次,构建结合通道–时 间注意力机制(Channel–Time Attention, CTA)的双向长短期记忆(Bidirectional Long Short Term Memory, BiLSTM)网络作为解码模型。结果表明,该模型平均解码准确率 为67.50%,并通过对比分析验证了CTA-BiLSTM解码模型的有效性。 3. 搭建了在线言语想象实验平台,以解决离线字符解码的局限性。首先,设计了 直观且用户友好的界面,并建立了数据实时采集及缓存模块。其次,采用预处理方法 确保数据与模型兼容,并将信号解码为字符概率分布。最后,挑选出概率最高的字符 显示于界面,并采用语言模型对语句进行校正。结果表明,语言模型的使用显著提高 了单词准确率,还原了被试者言语意图,验证了该平台的有效性和实用性。 本文通过构建字符和句子EEG信号数据集,验证了字符信号的可分性,并利用解 码模型将大脑活动映射为字符输出。同时,搭建在线平台,为实时BCI系统开发提供 了参考,并为未来BCI技术发展与应用奠定了基础。 |
论文外文摘要: |
Brain Computer Interface (BCI) technology can reconstruct speech intention by interpret ingspeechimageryneuralsignalsinthebrain, helpingindividualswithseverespeechgeneration disorders recover communication skills. This paper verifies the separability of discrete collect ed character signals, and labels and decodes continuously collected sentence signals to achieve online text generation and real-time correction. The main research content is as follows: 1. This paper constructs a character level electroencephalogram (EEG) signal dataset, s electing 26 English letters and 5 punctuation marks as the imaginative materials. Firstly, a method combining Wavelet Scattering Transform (WST) with Kernel Principal Component Analysis (KPCA) is proposed to extract time-frequency features. Secondly, using the Extreme Gradient Boosting (XGBoost) classifier to classify character signals, the average accuracy is 77.89%. Finally, signal separability was verified through further clustering analysis and com parative studies. 2. This paper constructs a sentence level EEG dataset and collected signals through con tinuous imagination to address the limitations of discrete signal acquisition. Firstly, Dynamic TimeWarping(DTW)andHiddenMarkovModel(HMM)areusedfortimecalibrationandlabel labeling. Secondly, a Bidirectional Long Short Term Memory (BiLSTM) network combining Channel Time Attention (CTA) mechanism is constructed as the decoding model. The results showed that the average decoding accuracy of the model was 67.50%, and the effectiveness of the CTA-BiLSTM decoding model was verified through comparative analysis. 3. This paper establishes an online speech imagination experimental platform to address the limitations of offline character decoding. Firstly, an intuitive and user-friendly interface was designed, and a real-time data collection and caching module was established. Secondly, preprocessing methods are used to ensure compatibility between the data and the model, and the signal is decoded into a character probability distribution. Finally, select the character with the highest probability to display on the interface, and use a language model to correct the statemen t. The results showed that the use of language models significantly improved word accuracy, restored the speech intentions of participants, and verified the effectiveness and practicality of the platform. This paper verifies the separability of character signals by constructing character and sen tence EEG signal datasets, and uses a decoding model to map brain activity to character output. At the same time, building an online platform provides a reference for the development of real time BCI systems and lays the foundation for the future development and application of BCI technology. |
中图分类号: | TP391 |
开放日期: | 2025-06-20 |