There is a connection between gradient descent based optimizers and the dynamics of damped harmonic oscillators. What does that mean? We now have a better theory for optimization algorithms.
In this episode I explain how all this works.
All the formulas I mention in the episode can be found in the post The physics of optimization algorithms
Enjoy the show.
WARNING!! Neural networks can memorize secrets (ep. 100)
Attacks to machine learning model: inferring ownership of training data (Ep. 99)
Don't be naive with data anonymization (Ep. 98)
Why sharing real data is dangerous (Ep. 97)
Building reproducible machine learning in production (Ep. 96)
Bridging the gap between data science and data engineering: metrics (Ep. 95)
A big welcome to Pryml: faster machine learning applications to production (Ep. 94)
It's cold outside. Let's speak about AI winter (Ep. 93)
The dark side of AI: bias in the machine (Ep. 92)
The dark side of AI: metadata and the death of privacy (Ep. 91)
The dark side of AI: recommend and manipulate (Ep. 90)
The dark side of AI: social media and the optimization of addiction (Ep. 89)
More powerful deep learning with transformers (Ep. 84) (Rebroadcast)
How to improve the stability of training a GAN (Ep. 88)
What if I train a neural network with random data? (with Stanisław Jastrzębski) (Ep. 87)
Deeplearning is easier when it is illustrated (with Jon Krohn) (Ep. 86)
[RB] How to generate very large images with GANs (Ep. 85)
More powerful deep learning with transformers (Ep. 84)
[RB] Replicating GPT-2, the most dangerous NLP model (with Aaron Gokaslan) (Ep. 83)
What is wrong with reinforcement learning? (Ep. 82)
Create your
podcast in
minutes
It is Free
Insight Story: Tech Trends Unpacked
Zero-Shot
Fast Forward by Tomorrow Unlocked: Tech past, tech future
The Unbelivable Truth - Series 1 - 26 including specials and pilot
Elliot in the Morning