基于稀疏贝叶斯的Dirichlet问题求解方法

杨东华; YANG Donghua; 莫艳; MO Yan

doi:10.48014/fcpm.20240312002

基于稀疏贝叶斯的Dirichlet问题求解方法

Sparse Bayesian Based Method to Solve Dirichlet Problem

中国理论数学前沿 2024年第2卷第2期页码[5-15] 下载全文[5.1MB] [HTML]

摘要

偏微分方程与许多实际问题有着密切的联系, 由于其通常不易获得解析解, 其数值解法是众多学者研究的热门课题。稀疏贝叶斯学习是近年来机器学习的研究热点。相关向量机 (Relevance Vector Machine, RVM) 是一种基于稀疏贝叶斯学习的学习机, 在解决逼近问题上表现出良好的性能。本文基于再生核理论和相关向量机, 提出了一种求解离散及非离散Dirichlet问题的新方法。基于再生核理论构造满足方程条件的再生核空间, 把方程求解问题转化为相应的目标优化问题, 然后在相关向量机框架中进行求解, 得到以再生核线性组合稀疏表示的近似解。相比于传统方法, 本文方法不需要画网格; 相较于一般的机器学习方法, 本文方法基于对超参数的处理具有出色的避免过拟合的能力, 能够计算出结果的后验概率分布以提供不确定性量化, 不需要交叉验证过程设置额外的正则化参数, 减少了模型的复杂度, 提高了模型的泛化能力、可解释性和效率。稀疏性分析说明了所提方法的稀疏性质和扩展到高维问题的可能性。与常用的有限差分法、神经网络方法以及支持向量机方法的对比实验结果表明, 所提方法具有较好的稀疏性、鲁棒性以及有效性。

Abstract

Partial differential equations are closely related to many practical problems. Since they are usually difficult to obtain analytical solutions of partial differential equations, their numerical solution is a hot topic for many scholars. Sparse Bayesian learning is a hot research spot of machine learning in recent years. Relevance Vector Machine (RVM) is a learning machine based on sparse Bayesian learning, which shows good performance in solving approximation problems. In this paper, based on the reproducing kernel theory and relevance vector machine, a new method for solving discrete and non-discrete dirichlet problems is proposed. Based on the reproducing kernel theory, the reproducing kernel space satisfying the equation conditions is constructed, and the equation solving problem is transformed into the corresponding objective optimization problem. Then, the problem is solved in the framework of correlation vector machine, and the approximate solution is obtained by sparse representation of linear combination of reproducing kernels. Compared with the traditional method, this method does not need to draw the grid. Compared with the general machine learning method, this method in this paper has excellent ability to avoid over-fitting based on the processing of hyperparameters. It can calculate the posterior probability distribution of the results to provide uncertainty quantification. It does not need to set additional regularization parameters in the cross-validation process, which reduces the complexity of the model and improves the model’s generalization ability, interpretability and efficiency. Sparsity analysis shows the sparse nature of the proposed method and the possibility of extending it to high-dimensional problems. Comparison experiments with the commonly used finite difference method, neural network method and support vector machine method show that the proposed method has better sparsity, robustness and effectiveness.

DOI

10.48014/fcpm.20240312002

文章类型

研究性论文

收稿日期

2024-03-12

接收日期

2024-04-07

出版日期

2024-06-28

关键词

Dirichlet问题, 稀疏贝叶斯, 相关向量机, 高斯再生核Hilbert空间, 稀疏解

Keywords

Dirichlet problem, sparse Bayesian, Relevance Vector Machine, Gaussian-RKHS, sparse solution

作者

杨东华, 莫艳^*

Author

YANG Donghua, MO Yan^*

所在单位

广东工业大学数学与统计学院, 广州510520

Company

School of Mathematics and Statistics, Guangdong University of Technology, Guangzhou 510520, China

浏览量

345

下载量

203

基金项目

广州市科技计划(202102020704)

参考文献

[1] SMITH G D. Numerical Solution of Partial Differential Equations: Finite Difference Methods[M]. Clarendon Press, Oxford, 1978.
[2] 李荣华. 偏微分方程数值解[M]. 北京: 高等教育出版社, 2007.
[3] LAGARIS I E, LIKAS A C. Artificial neural networks for solving ordinary and partial differential equations[J]. IEEE Transactions on Neural Networks, 1998, 9(5): 987-1000.
[4] KEVIN S. MCFAll, JAMES R. MAHAN. Artificial Neural Network Method for Solution of Boundary Value Problems with Exact Satisfaction of Arbitrary Boundary Conditions[J]. IEEE Transactions on Neural Networks, 2009, 20(8): 1221-1233.
https://doi.org/10.1109/TNN.2009.2020735
[5] JUSTIN A SIRIGNANO, KONSTANTINOS SPILIOPOULOS. A deep learning algorithm for solving partial differential equations[J]. Computational Physics, 2018, 375: 1339-1364.
https://doi.org/10.1016/j.jcp.2018.08.029
[6] MAZIAR RAISSI, PARIS PERDIKARIS, GEORGE EM KARNIADAKIS. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[J]. Computational Physics, 2019, 378: 686-707.
https://doi.org/10.1016/j.jcp.2018.10.045
[7] MAZIAR RAISSI, ALIREZA YAZDANI, GEORGE EM KARNIADAKIS. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations[J]. Science, 2020, 367: 1026-1030.
https://doi.org/10.1126/science.aaw4741
[8] JEREMY YU, LU LU, XUHUI MENG, et al. Gradientenhanced physics-informed neural networks for forward and inverse PDE problems[J]. Computer Methods in Applied Mechanics and Engineering, 2022, 393: 114823.
https://doi.org/10.1016/j.cma.2022.114823
[9] SAITOH S. Integral transforms, reproducing Kernels and their applications[M]. in: Chapman & Hall/CRC Research Notes in Mathematics Series 369. CRC Press, 1997.
https://doi.org/10.1201/9781003062301
[10] SAITOH S. Approximate real inversion formulas of the gaussian convolution[J]. Applicable Analysis, 2004, 83(7): 727-733.
https://doi.org/10.1080/00036810410001657198
[11] MATSUURA T, SAITOH S. Dirichlet principle using computers[J]. Applicable Analysis, 2005, 84(10): 989-1003.
https://doi.org/10.1080/00036810412331297208
[12] MO Y, QIAN T. Support vector machine adapted Tikhonov regularization method to solve Dirichlet problem[J]. Applied Mathematics & Computation, 2014, 245: 509-519.
https://doi.org/10.1016/j.amc.2014.07.089
[13] 王丹荣, 莫艳. 基于支持向量机的离散线性微分方程求解方法[J]. 广东工业大学学报, 2020, 37(02): 87-93.
https://doi.org/10.12052/gdutxb.190050
[14] SORIA O E, GOMEZ S J, MARTIN J D. BELM: Bayesian Extreme Learning Machine[J]. IEEE Transactions on Neural Networks, 2011, 22(3): 505-509.
https://doi.org/10.1109/TNN.2010.2103956
[15] TIPPING M E. Bayesian Inference: An Introduction to Principles and Practice in Machine Learning[C]. Advanced Lectures on Machine Learning, Ml Summer Schools, Canberra, Australia, February, Tübingen, Germany, August, Revised Lectures. Springer Berlin Heidelberg, 2004.
https://doi.org/10.1007/978-3-540-28650-9_3
[16] TIPPING M E. The Relevance Vector Machine[J]. In Proceedings of the 12th International Conference on Neural Information Processing Systems(NIPS'99), 1999, MIT Press, Cambridge, MA, USA, 652-658.
https://doi.org/10.5555/3009657.3009750
[17] TIPPING M E. Sparse Bayesian Learning and the Relevance Vector Machine[J]. Journal of Machine Learning Research, 2001, 1(3): 211-244.
https://doi.org/10.1162/15324430152748236
[18] XU X M, MAO Y F, XIONG J N, et al. Classification Performance Comparison between RVM and SVM[C]. 2007 International Workshop on Anti-Counterfeiting, Security and Identification(ASID), 2007: 208-211.
https://doi.org/10.1109/IWASID.2007.373728
[19] HA QUANG MINH. Some properties of gaussian reproducing kernel Hilbert spaces and their implications for function approximation and learning theory[J]. Constructive Approximation, 2010, 32(2): 307-338.
https://doi.org/10.1007/s00365-009-9080-0
[20] FAUL C ANITA, TIPPING E MICHAEL. Analysis of Sparse Bayesian Learning[J]. Neural Information Processing Systems, 2002: 383-390.
https://doi.org/10.5555/2980539.2980590
[21] LUO J, VONG C M, WONG P K. Sparse Bayesian Extreme Learning Machine for Multi-classification[J]. IEEE Transactions on Neural Networks & Learning Systems, 2017, 25(4): 836-843.
https://doi.org/10.1109/TNNLS.2013.2281839

引用本文

杨东华, 莫艳. 基于稀疏贝叶斯的Dirichlet问题求解方法[J]. 中国理论数学前沿, 2024, 2(2): 5-15.

Citation

YANG Donghua, MO Yan. Sparse Bayesian based method to solve Dirichlet problem[J]. Frontiers of Chinese Pure Mathematics, 2024, 2(2): 5-15.