脑机接口是大脑与计算机融合感知的一项新技术，通过在大脑与外部辅助设备之间建立连接方式并对事物进行控制。脑电模态刺激源识别是实现脑接口交互作用的关键步骤。本文以视听觉脑电信号的采集、去噪、特征提取和分类识别为研究内容，实现脑电视听模态刺激源识别。主要的研究工作如下：

原始采集的视听觉脑电信号中含有大量噪声，针对经验模态分解算法去除脑电信号噪声过程中存在模态混淆现象的问题，提出了自适应经验模态分解与快速独立成分分析结合的去噪算法，实现脑电信号中噪声的有效去除。

采用平稳小波共空间算法以及改进的多元多尺度熵算法进行特征提取。通过平稳小波共空间算法获取脑电信号频率、能量的特征信息。同时，针对多元尺度熵算法计算效率不足的问题，提出粗粒化计算添加多元延时矢量的方法，得到不同刺激在不同尺度下的熵值。

针对LeNet5网络对视听模态刺激源特征表征单一的问题，提出集成Inception v1的LeNet5并联卷积神经网络结构。改进的网络结构具有不同大小卷积核的多个并行卷积层，实现网络结构的宽度扩大，将改进卷积神经网络用于脑电刺激源视听模态识别，以达到更高的分类识别的准确率。

结果表明，改进去噪算法的信噪比、均方根误差、相关系数相对于小波去噪、独立成分去噪分别提升了约为0.05-0.14%、0.18-0.23%、0.1-0.26%。改进多元多尺度熵算法的识别准确率高于平稳小波共空间算法。在分类识别器构造上，改进的卷积神经网络的准确率高于线性分类器和支持向量机，视觉、听觉、视听觉混合刺激识别精度最高可达87.98%、88.4%、90.12%，视听觉混合刺激的识别效果比单一刺激的识别效果明显更有优势。本研究对实现脑机接口系统具有一定的应用价值。

Brain-Computer Interface is a new technology of brain and computer fusion perception, which establishes a connection between the brain and external auxiliary equipment and controls things. Recognition of brain electrical stimulation source is a key step to realize the interaction of brain-computer interface. This thesis takes the collection, denoising, feature extraction and classification of audiovisual EEG signals as the research content, and realizes the recognition of brain-television audio modal stimuli. The main research work is as follows:

(1) The original audio-visual EEG signal contains a lot of noise. Aiming at the problem of mode confusion in the process of removing the noise of EEG signal by empirical mode decomposition, a denoising algorithm combining adaptive EMD and fast independent component analysis  is proposed to realize the effective removal of noise in EEG signal.

(2) The stationary wavelet common space algorithm and the improved multivariate multi-scale entropy algorithm are used for feature extraction. The characteristic information of frequency and energy of EEG signal is obtained by stationary wavelet common space algorithm. At the same time, aiming at the problem of low computational efficiency of multi-scale entropy algorithm, a coarse-grained calculation method with multi delay vector is proposed to obtain the entropy values of different stimuli at different scales.

(3) To solve the problem of single feature representation of audio-visual modal stimuli by LeNet5 network, a parallel convolutional neural network structure of LeNet5 integrated with Inception v1 is proposed. The improved convolution neural network structure has multiple parallel convolution layers with different convolution cores to expand the width of the network structure. The improved convolution neural network is used for audio-visual modal recognition of EEG stimuli to achieve higher classification accuracy.

The results show that the signal-to-noise ratio, root mean square error, and correlation coefficient of the improved denoising algorithm are improved by about 0.05-0.14%, 0.18-0.23%, and 0.1-0.26% respectively compared with wavelet denoising and independent component denoising. The recognition accuracy of the improved multivariate multiscale entropy algorithm is higher than that of the stationary wavelet co-space algorithm. In the structure of the classification recognizer, the accuracy of the improved convolutional neural network is higher than that of the linear classifier and support vector machine. The recognition accuracy of visual, auditory, and audiovisual mixed stimuli can reach up to 87.98%, 88.4%, and 90.12%. The recognition effect of the mixed stimulus is obviously more advantageous than the recognition effect of the single stimulus. This research has certain application value for the realization of BCI system.