本文针对回声状态网络不稳定特性以及最优储备池结构设计的难题，提出了 L1 正 则化的内在可塑性回声状态网络，解决了回声状态网络不稳定和储备池规模难以和具体 任务相匹配的问题，然后提出了线性偏离，并使用线性偏离对 L1 正则化求解输出权值 进行优化，最后对瓦斯浓度预测过程进行应用。本文主要研究内容如下：

首先，针对回声状态网络储备池不稳定以及网络规模难以和具体任务相匹配的问题， 提出一种基于 L1 正则化的内在可塑性回声状态网络优化方法。通过神经元内在可塑性 实现储备池神经元激活函数的调节，使网络更加稳定健壮，然后使用 L1 范数正则化求解输出权值，利用 L1 正则化的选择功能控制储备池规模大小，避免模型出现过拟合。 通过 Mackey-Glass 和 Lorenz 动力系统预测实验表明，本文所提的 L1 正则化的内在可塑 性网络在保证网络稳定的前提下，不仅克服了网络规模过大或过小导致网络模型过拟合 或欠拟合的缺陷，而且提高了模型求解的数值稳定性以及控制了网络的复杂程度。

其次，针对 L1 正则化的内在可塑性回声状态网络模型训练时间长以及回声状态网 络储备池机理不明确的问题，提出一种基于线性偏离和 L1 正则化的内在可塑性回声状 态网络。该网络提出了线性偏离的概念，用来度量储备池和神经元的非线性程度，使用 储备池线性偏离给内在可塑性回声状态网络使储层稳健提供合理性解释，并使用神经元 线性偏离筛选神经元节点对 L1 正则化求解网络输出权值进行优化，以此提高模型训练 速度。在混沌数据集以及瓦斯浓度数据集中的实验结果表明，本文所提出的方法不仅提 高了回声状态网络的稳定性和预测能力，还能够有效的缩短模型训练时间，且本文提出 的线性偏离对储备池理论研究可以提供帮助。

最后，在基于线性偏离和 L1 正则化的内在可塑性回声状态网络算法研究的基础上， 设计并实现了一套基于 Browser/Server 架构的瓦斯浓度预测管理系统，主要实现了瓦斯 浓度和风险等级的判断功能，并对结果实现可视化界面展示，具有重要的实用价值。

In this paper, aiming at the unstable characteristics of the echo state network and the difficulty of designing the optimal reserve pool structure, the intrinsic plasticity of the L1 regularization echo state network is proposed to solve the problems of the instability of the echo state network and the difficulty of matching the size of the reserve pool with specific tasks. Then, the deviation linear is proposed, and the deviation linear is used to optimize the output weight of L1 regularization solution. Finally, the prediction process of gas concentration is applied. The main research contents of this paper are as follows:

Firstly, aiming at the instability of the echo state network reservoir and the difficulty of matching the network scale with the specific task, an intrinsic plastic echo state network optimization method based on L1 regularization was proposed. The activation function of the reservoir neurons is adjusted through the intrinsic plasticity of neurons, so that the network is more stable and robust. Then L1 norm regularization is used to solve the output weights, and the selection function of L1 regularization is used to control the size of the reservoir to avoid overfitting of the model. The Mackey-Glass and Lorenz dynamic system prediction experiments show that the proposed L1 regularized intrinsic plastic network not only overcomes the defects of overfitting or underfitting of the network model caused by too large or too small network scale, but also ensures the stability of the network. Moreover, it improves the numerical stability of the model solution and controls the complexity of the network.

Secondly, aiming at the problems of long training time and unclear mechanism of echo state network reserve pool of L1 regularization, an intrinsic plasticity echo state network based on deviation linear and L1 regularization was proposed. The concept of deviation linear is proposed to measure the degree of nonlinearity of the reserve pool and neurons. The deviation linear of the reserve pool is used to provide a rational explanation for the intrinsic plasticity echo state network to make the reservoir robust, and the deviation linear of neurons is used to screen neuron nodes to optimize the output weight of the L1 regularization solution network, so as to improve the model training speed. The experimental results in chaotic data sets and gas concentration data sets show that the proposed method not only improves the stability and prediction ability of echo state network, but also effectively reduces the training time of the model. Moreover, the linear deviation proposed in this paper can provide help for the theoretical research of reserve pool.

Finally, on the basis of the research of intrinsic plasticity echo state network algorithm based on linear deviation and L1 regularization, a set of gas concentration prediction management system based on Browser/Server architecture is designed and implemented, which mainly realizes the judgment function of gas concentration and risk level, and realizes the visual interface display of results, which has important practical value.

[1]Hutson M. AI Glossary: Artificial intelligence, in so many words[J]. Science, 2017, 357(6346): 19-19.

[2]David C. China enters the battle for AI talent[J]. Nature, 2018, 553(7688): 260-261.

[3]Zhou Y, Cao R. The Artificial Neural Network Prediction Algorithm Research of Rail-gun Current and Armature Speed Based on B-dot probes array[J]. Measurement, 2018, 133(2134): 47-55.

[4]张伟, 李扬, 张吉等. 融合时空行为与社交关系的用户轨迹识别模型[J]. 计算机学报, 2021, 44(11): 2173-2188.

[5]张杰, 王川, 董晓霞等. 基于裂变模式的农业农村信息快速调查及分析平台开发与应用[J]. 中国农业科学, 2022, 55(21): 4158-4174.

[6]Bayro-Corrochano E, Solis-Gamboa S, Altamirano-Escobedo G, et al. Quaternion Spiking and Quaternion Quantum Neural Networks: Theory and Applications[J]. International Journal of Neural Systems, 2021, 31(02): 205-214.

[7]李杰, 李响, 许元铭等. 工业人工智能及应用研究现状及展望[J]. 自动化学报, 2020, 46(10): 2031-2044.

[8]王雨露, 李飞, 杨震等. 基于深度前馈神经网络的多因子人体表面积计算模型[J]. 计算机工程与科学, 2023, 45(01): 119-126.

[9]Jiang J, Wang H, Xie J, et al. Medical knowledge embedding based on recursive neural network for multi-disease diagnosis[J]. Elsevier, 2020, 103: 47-55.

[10]胡敏, 高永, 吴昊等. 融合边缘检测和递归神经网络的视频表情识别[J]. 电子测量与仪器学报, 2020, 34(07): 103-111.

[11]Miao P, Shen Y, Li Y, et al. Finite-time recurrent neural networks for solving nonlinear optimization problems and their application[J]. Neurocomputing, 2016, 177: 120-129.

[12]耿磊, 傅洪亮, 陶华伟等. 基于动态卷积递归神经网络的语音情感识别[J]. 计算机工程, 2023, 49(04): 125-137.

[13]Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult[J]. IEEE transactions on neural networks, 1994, 5(2): 157-166.

[14]Han S I, Lee J M. Fuzzy echo state neural networks and funnel dynamic surface control for prescribed performance of a nonlinear dynamic system[J]. IEEE Transactions on Industrial Electronics, 2013, 61(2): 1099-1112.

[15]Maass W, Natschläger T, Markram H. Real-time computing without stable states: A new framework for neural computation based on perturbations[J]. Neural computation, 2002, 14(11): 2531-2560.

[16]Wang H, Lei Z, Liu Y, et al. Echo state network based ensemble approach for wind power forecasting[J]. Energy Conversion and Management, 2019, 201: 112188. DOI: 10.1016/j.enconman.2019.112188.

[17]Fuji Ren, Yindong Dong, Wei Wang. Emotion recognition based on physiological signals using brain asymmetry index and echo state network[J]. Neural Computing and Applications, 2019, 31(9): 4491-4501.

[18]Yu Cao, Jian Huang. Neural-Network-Based Nonlinear Model Predictive Tracking Control of a Pneumatic Muscle Actuator-Driven Exoskeleton[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 7(06): 1478-1488.

[19]Jaeger H. Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication[J]. Science, 2004, 304(5667): 78-80.

[20]Ferreira A A, Ludermir T B, Aquino R. An approach to reservoir computing design and training[J]. Expert Systems with Applications, 2013, 40(10): 4172-4182.

[21]Lun S X, Yao X S, Hu H F. A New Echo State Network with Variable Memory Length[J]. Information Sciences, 2016, 370(20): 103-119.

[22]王林, 王燕丽, 安泽远. 改进粒子群算法优化回声状态网络的电力需求预测研究[J]. 计算机工程与科学, 2022, 44(08): 1457-1466.

[23]Bala A, Ismail I, Ibrahim R, et al. Prediction using cuckoo search optimized echo state network[J]. Arabian Journal for Science and Engineering, 2019, 44(11): 9769-9778.

[24]张昭昭, 朱应钦, 乔俊飞等. 一种基于行为空间的回声状态网络参数优化方法[J]. 信息与控制, 2021, 50(5): 556-565.

[25]Triesch J. A gradient rule for the plasticity of a neuron’s intrinsic excitability[C] //Artificial Neural Networks: Biological Inspirations - ICANN 2005: 15th International Conference, Warsaw, Poland, September 11-15, 2005. Proceedings, Part I 15. Springer Berlin Heidelberg, 2005: 65-70.

[26]Schrauwen B, Wardermann M, Verstraeten D, et al. Improving reservoirs using intrinsic plasticity[J]. Neurocomputing, 2008, 71(7-9): 1159-1171.

[27]Wang X, Jin Y, Hao K. Synergies between synaptic and intrinsic plasticity in echo state networks[J]. Neurocomputing, 2021, 432: 32-43.

[28]Jaeger H. Discovering multiscale dynamical features with hierarchical echo state networks[R]. Jacobs University Bremen: Technical Reports, 2007.

[29]Xue Y, Yang L, Haykin S. Decoupled echo state networks with lateral inhibition[J]. Neural Networks, 2007, 20(3): 365-376.

[30]Boccato L, Attux R, Zuben F V. Self-organization and lateral interaction in echo state network reservoirs[J]. Neurocomputing, 2014, 138: 297–309.

[31]Ma Q L, Chen W B. Modular state space of echo state network[J]. Neurocomputing, 2013, 122: 406-417.

[32]Qiao J, Li F, Han H, et al. Growing echo-state network with multiple subreservoirs[J]. IEEE transactions on neural networks and learning systems, 2016, 28(2): 391-404.

[33]王磊, 乔俊飞, 杨翠丽等. 基于灵敏度分析的模块化回声状态网络修剪算法[J]. 自动化学报, 2019, 45(6): 1136-1145.

[34]Gallicchio C, Micheli A. Architectural and Markovian factors of echo state networks[J]. Neural Networks the Official Journal of the International Neural Network Society, 2011, 24(5): 440-456.

[35]Kawai Y, Park J, Asada M. A small-world topology enhances the echo state property and signal propagation in reservoir computing[J]. Neural Networks, 2019, 112: 15-23.

[36]Zhou Y, Zhang M, Lin K P. Time series forecasting by the novel gaussian process wavelet self-join adjacent-feedback loop reservoir model[J]. Expert Systems with Applications, 2022, 198: 116772.

[37]Dutoit X, Schrauwen B, Campenhout J V, et al. Pruning and regularization in reservoir computing[J]. Neurocomputing, 2009, 72(9): 1534-1546.

[38]Chatzis S P, Demiris Y. Echo state Gaussian process[J]. IEEE Transactions on Neural Networks, 2011, 22(9): 1435-1445.

[39]韩敏, 任伟杰, 许美玲. 一种基于L1范数正则化的回声状态网络[J]. 自动化学报, 2014, 40(11): 2428-2435.

[40]Zhang L, Hua C, Tang Y, et al. Ill-posed echo state network based on L-curve method for prediction of blast furnace gas flow[J]. Neural Processing Letters, 2016, 43(1): 97-113.

[41]Løkse S, Bianchi F M, Jenssen R. Training echo state networks with regularization through dimensionality reduction[J]. Cognitive Computation, 2017, 9: 364-378.

[42]许美玲. 基于回声状态网络的多元时间序列预测研究[D]. 大连: 大连理工大学, 2016.

[43]沈力华, 陈吉红, 曾志刚等. 多稀疏回声状态网络预测模型[J]. 控制理论与应用, 2018, 35(04): 421-428.

[44]Jaeger H. Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the" echo state network" approach[J]. 2002.

[45]肖勇, 赵云, 涂治东等. 基于改进的皮尔逊相关系数的低压配电网拓扑结构校验方法[J]. 电力系统保护与控制, 2019, 47(11): 37-43.

[46]Zhang W Y, Wei Z W, Wang B H, et al. Measuring mixing patterns in complex networks by Spearman rank correlation coefficient[J]. Physica A: Statistical Mechanics and its Applications, 2016, 451: 440-450.

[47]牛璐, 刘秀敏. 基于 Kendall 秩相关系数的稳健马氏距离及其应用[J]. 内蒙古大学学报: 自然科学版, 2020, 51(2): 123-128.

[48]Ozturk M C, Xu D, Principe J C. Analysis and design of echo state networks[J]. Neural computation, 2007, 19(1): 111-138.

[49]Wang S, Yang X J, Wei C J. Harnessing non-linearity by sigmoid-wavelet hybrid echo state networks (SWHESN)[C]//2006 6th World Congress on Intelligent Control and Automation. IEEE, 2006, 1: 3014-3018.

[50]逯程, 徐廷学, 王虹. 基于L1/2范数正则化的塑性回声状态网络故障诊断模型[J]. 北京航空航天大学学报, 2018, 44(03): 535-541.

[51]Gluhovsky I. Multinomial least angle regression[J]. IEEE transactions on neural networks and learning systems, 2011, 23(1): 169-174.

[52]Wang X, Jin Y, Hao K. Echo state networks regulated by local intrinsic plasticity rules for regression[J]. Neurocomputing, 2019, 351: 111-122.

[53]Stemmler M, Koch C. How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate[J]. Nature neuroscience, 1999, 2(6): 521-527.

[54]Zhang Z Z, Zhu Y Q, Wang X H, et al. Optimal echo state network parameters based on behavioural spaces[J]. Neurocomputing, 2022, 503: 299-313.

[55]王华秋, 王斌, 聂珍. 一种应用多储备池回声状态网络的图像语义映射研究[J]. 现代图书情报技术, 2015, 259(06): 41-48.

[56]何怡刚, 隋永波, Farhan Ali等. 自适应OFDM系统的时域信道预测研究[J]. 电子测量与仪器学报, 2021, 35(05): 100-110.

[57]Butcher J B, Verstraeten D, Schrauwen B, et al. Reservoir computing and extreme learning machines for non-linear time-series data analysis[J]. Neural networks, 2013, 38: 76-89.

[58]张景肖, 刘史诗, 王伟华等. 基于AIC准则的函数型数据主成分联合选择研究[J]. 数理统计与管理, 2022, 41(04): 610-622.

[59]牛胜锁, 张达, 梁志瑞等. 基于抗差总体最小二乘法的电力系统谐波状态估计[J]. 电力系统保护与控制, 2014, 42(11): 106-111.

[60]Watanabe S. A Widely Applicable Bayesian Information Criterion[J]. Journal of Machine Learning Research, 2012, 14(1): 867-897.

[61]王世元, 王文月, 钱国兵. 自适应矩估计最大相关熵算法的混沌序列预测[J]. 华南理工大学学报(自然科学版), 2019, 47(04): 20-34.

[62]Yang C, Qiao J, Han H, et al. Design of polynomial echo state networks for time series prediction[J]. Neurocomputing, 2018, 290: 148-160.

[63]王迪, 王萍, 石君志. 一种错误率可控的混沌时间序列区间预测算法[J]. 控制与决策, 2019, 34(05): 956-964.

[64]Li F R. Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties[J]. Publications of the American Statistical Association, 2001, 96(456): 1348-1360.

[65]Efron B, Hastie T, Tibshirani J R. Least Angle Regression[J]. Annals of Statistics, 2004, 32(2): 407-451.

[66]李柏松, 任伟杰, 韩敏. 基于HSIC-GL的多元时间序列非线性Granger因果关系分析[J]. 信息与控制, 2021, 50(03): 356-365.

[67]Soh H, Demiris Y. Spatio-temporal learning with the online finite and infinite echo-state Gaussian processes[J]. IEEE transactions on neural networks and learning systems, 2014, 26(3): 522-536.

[68]李冬, 彭苏萍, 杜文凤等. 煤层瓦斯突出危险区综合预测方法[J]. 煤炭学报, 2018, 43(02): 466-472.