Robin Pre-Training for the Deep Ritz Method


  • Luca Courte
  • Marius Zeinhofer Simula Research Laboratory



Deep Ritz Method, Neural Networks, Variational Problems, Essential Boundary Conditions


We analyze the training process of the Deep Ritz Method for elliptic equations with Dirichlet boundary conditions and highlight problems arising from essential boundary values. Typically, one employs a penalty approach to enforce essential boundary conditions, however, the naive approach to this problem becomes unstable for large penalizations. A novel method to compensate this problem is proposed, using a small penalization strength to pre-train the model before the main training on the target penalization strength is conducted. We present numerical evidence that the proposed method is beneficial.


M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), pages 265–283, 2016.

I. Babuˇska. The finite element method with penalty. Mathematics of computation, 27(122): 221–228, 1973.

E. C. Cyr, M. A. Gulian, R. G. Patel, M. Perego, and N. A. Trask. Robust Training and Initialization of Deep Neural Networks: An adaptive Basis Viewpoint. In Mathematical and Scientific Machine Learning, pages 512–536. PMLR, 2020.

M. Dissanayake and N. Phan-Thien. Neural-Network-based Approximations for solving Partial Differential Equations. Communications in Numerical Methods in Engineering, 10 (3):195–201, 1994.

W. E and B. Yu. The Deep Ritz method: a Deep Learning-based numerical Algorithm for solving Variational Problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018. doi: 10.1007/s40304-018-0127-z. URL 10.1007/s40304-018-0127-z.

I. E. Lagaris, A. Likas, and D. I. Fotiadis. Artificial Neural Networks for solving Ordinary and Partial Differential Equations. IEEE transactions on neural networks, 9(5):987–1000, 1998. doi: 10.1109/72.712178. URL

K. Lee, N. A. Trask, R. G. Patel, M. A. Gulian, and E. C. Cyr. Partition of unity networks: Deep hp-approximation. In CEUR Workshop Proceedings, volume 2964, page 180. CEURWS, 2021.

Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for parametric partial differential equations. International Conference on Learning Representations, 2021.

J. Mueller and M. Zeinhofer. Error Estimates for the Deep Ritz Method with

Boundary Penalty. In 3rd Conference on Mathematical and Scientific Machine Learning (MSML2022), 2022.

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.

M. Raissi and G. E. Karniadakis. Hidden physics models: Machine learning of nonlinear partial differential equations. Journal of Computational Physics, 357:125–141, 2018. doi: 10.1016/ URL

J. Sirignano and K. Spiliopoulos. DGM: A Deep Learning Algorithm for solving Partial Differential Equations. Journal of computational physics, 375:1339–1364, 2018. doi: 10.1016/ URL

R. van der Meer, C. W. Oosterlee, and A. Borovykh. Optimally weighted loss functions for solving pdes with neural networks. Journal of Computational and Applied Mathematics, 405:113887, 2022. doi: 10.1016/ URL

S. Wang, Y. Teng, and P. Perdikaris. Understanding and mitigating gradient flow

pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021. doi: 10.1137/20M1318043. URL