Скачать или смотреть Neural Network + Differential Equations = Universal Differential Equations

Colab code: https://colab.research.google.com/dri...

Miro notes: https://miro.com/app/board/uXjVIYvo_n...

*****

Neural Networks can make predictions that violate basic physics or laws of thermodynamics if aimed only at minimizing a loss function.



To fix this issue, ML scientists introduced PINNs - Physics Informed Neural Networks - where you penalize a neural network when it makes physically nonsense predictions.



But what if you don’t know the full physics of a system? How do you penalize the neural network in that case?



Universal Differential Equations (UDE) is the answer.



I am writing this article in praise of this marvelous technique that is truly changing the way we are looking at how to bring science and ML together. Even a popular domain is emerging as a result: Scientific Machine Learning (SciML).



Let us look at a spring-mass-damper system - a classic example in physics and engineering. Usually, it goes like this: mx''+bx'+kx=0



In a perfect world, these parameters m, c, k would be constants we measure in a lab.



But in real life, your damper might behave non-linearly. So you may not know what the damping force is.



That is where we can bring Universal Differential Equations into the picture.



Instead of blindly trusting a neural network or strictly forcing your physical laws down the model’s throat, you merge them.



In short, a UDE says: “I know some of the physics. Let me put that in. The rest that I don’t know? That’s the chunk I will replace with a neural network.”



So how do we do it with the spring–mass–damper? A hybrid model: Part physics, part neural network.



We know there is a second-order ODE term to account for acceleration and a ‘kx’ term for spring force.



However, suppose, we suspect the damping force is not the usual linear form. Maybe it is more complicated, or partially unknown.



mx''+kx+[unknown]=0



Now the “something unknown” becomes a learned function modeled by a neural network NN(θ).



[unknown] = NN(θ)



If you suspect a hidden/unknown effect, you can funnel that knowledge gap straight into the neural network term.



Note that here the neural network is predicting the damping term. We want to predict displacement x(t).



What does the UDE predict?



The “neural network” alone is not the UDE. Because the UDE has to predict x(t) so that you can compare the predicted x(t) with experimental x(t) and define the loss.



So how exactly does UDE predict x(t)?



1) Initial condition and experimental data fed to NN(θ)

2) Neural Network NN(θ) for the unknown term predicts damping

3) Combine with the known ODE: mx''+kx+NN(θ)=0

4) Numerical integration to predict x and x'

5) Compare predictions to experimental data

6) Back-propagation and optimization till you minimize the loss



You have the final UDE model.



I have made a lecture video on UDEs (for absolute beginners) on Vizuara’s YouTube channel. Do check this out. I hope you enjoy watching this lecture as much as I enjoyed making it: