Reverse Mode Autodiff in Python (general compute graph)

Pubblicato il: 10 giugno 2024
sul canale di: Machine Learning & Simulation

2,031

Let's implement automatic differentiation (=backpropagation) for a general directed acyclic compute graph to compute the gradient of a scalar-valued loss function. Here is the notebook: https://github.com/Ceyron/machine-lea...

-------

👉 This educational series is supported by the world-leaders in integrating machine learning and artificial intelligence with simulation and scientific computing, Pasteur Labs and Institute for Simulation Intelligence. Check out https://simulation.science/ for more on their pursuit of 'Nobel-Turing' technologies (https://arxiv.org/abs/2112.03235 ), and for partnership or career opportunities.

-------

📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-lea...

📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: / felix-koehler and / felix_m_koehler

💸 : If you want to support my work on the channel, you can become a Patreon here: / mlsim

🪙: Or you can make a one-time donation via PayPal: https://www.paypal.com/paypalme/Felix...

-------

Timestamps:
00:00 Intro
01:30 Describe a function by a compute graph
02:45 Symbolic Derivative
02:57 Implementing analytic functions
04:00 Manual definition of compute graph
06:23 A (primal) function library
07:42 Primal forward execution of the compute graph
13:38 Library of primitive with pullbacks
19:40 Implementing reverse-mode AD via a vJp transformation
27:50 Testing the AD
29:07 Syntactic Sugar
30:39 Outro

In questa pagina del sito puoi guardare il video online Reverse Mode Autodiff in Python (general compute graph) della durata di ore minuti seconda in buona qualità , che l'utente ha caricato Machine Learning & Simulation 10 giugno 2024, condividi il link con amici e conoscenti, su youtube questo video è già stato visto 2,031 volte e gli è piaciuto 93 spettatori. Buona visione!