Newton Informed Neural Operator: A Novel Machine Learning Approach for Computing Multiple Solutions of Nonlinear Partials Differential Equations

Neural networks have been widely used to solve partial differential equations (PDEs) in different fields, such as biology, physics, and materials science. Although current research focuses on PDEs with a singular solution, nonlinear PDEs with multiple solutions create a major problem. Different neural network methods including PINN, the Deep Ritz method, and DeepONet, are developed to handle PDEs but they can learn only one solution for a learning process. Due to multiple solutions, the problem becomes ill-posed, whereas operator learning tries to approximate the map between the unique solution and parameter functions in PDEs.

Function learning techniques can learn the solution function themselves, and neural networks in function learning are used to find the approximate solutions to PDEs. The function learning methods use Physics-Informed Neural Network (PINN)-based learning techniques to solve this problem, however, due to an ill-posed problem, the task becomes more difficult. Another existing method is Operator learning in which, multiple techniques are developed to solve PDEs. For example, DeepONet, FNO motivated by spectral methods, MgNO, HANO, and neural operator based on the transformer. All of these focus on the operator approximation between parameters and the solutions.

Researchers from Pennsylvania State University, USA, and King Abdullah University of Science and Technology, Saudi Arabia, proposed Newton Informed Neural Operator (NINO), a novel method to solve nonlinear PDEs with multiple solutions. NINO is developed on neural network techniques and is based on operator learning, which helps to capture many solutions in a single training process. This helps to overcome the challenges faced by the function learning methods in neural networks. Moreover, the classical Newton methods are integrated to improve the network architecture, ensuring better formulation of problems in operator learning.

After integration with traditional Newton methods, NINO learns multiple solutions efficiently in a single learning process using small data points compared to the neural network methods present. Moreover, researchers introduced two different training methods; the first method uses supervised data and utilizes the Mean Squared Error Loss (MSEL) as the primary optimization condition. The second method combines supervised and unsupervised learning and utilizes a hybrid function loss. This loss is integrated with MSEL for a small number of data with the ground truth, and with Newton’s loss for a large number of data without ground truth. 

The efficiency of the NINO is achieved by benchmarking both the methods, Newton solver and Neural operator, used during the experiment. The performance is evaluated in terms of total execution time, which includes the setup of matrices and vectors, GPU computation, and CUDA stream synchronization. Newton solver method uses 10 streams and CuPy with CUDA to parallelize the computation and completely utilize the GPU parallel processing capabilities to optimize the efficiency of execution time. On the other hand, the Neural operator method is naturally parallelized, completely using the GPU architecture without using multiple streams. 

In conclusion, researchers introduced Newton Informed Neural Operator (NINO), a novel method to solve nonlinear PDEs with multiple solutions. NINO can solve the problem faced by the function learning methods in neural networks. Also, researchers presented a theoretical analysis of the Neural operator method used during the experiment. This analysis shows that this method can efficiently learn the Newton operator and minimize the amount of supervised data needed. It learns solutions not available in the supervised learning data and can solve the problem in less time than traditional Newton methods.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our 43k+ ML SubReddit | Also, check out our AI Events Platform