Reverse Mode Autodiff in Python (general compute graph)

Veröffentlicht am: 10 Juni 2024
auf dem Kanal: Machine Learning & Simulation

2,031

Let's implement automatic differentiation (=backpropagation) for a general directed acyclic compute graph to compute the gradient of a scalar-valued loss function. Here is the notebook: https://github.com/Ceyron/machine-lea...

-------

👉 This educational series is supported by the world-leaders in integrating machine learning and artificial intelligence with simulation and scientific computing, Pasteur Labs and Institute for Simulation Intelligence. Check out https://simulation.science/ for more on their pursuit of 'Nobel-Turing' technologies (https://arxiv.org/abs/2112.03235 ), and for partnership or career opportunities.

-------

📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-lea...

📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: / felix-koehler and / felix_m_koehler

💸 : If you want to support my work on the channel, you can become a Patreon here: / mlsim

🪙: Or you can make a one-time donation via PayPal: https://www.paypal.com/paypalme/Felix...

-------

Timestamps:
00:00 Intro
01:30 Describe a function by a compute graph
02:45 Symbolic Derivative
02:57 Implementing analytic functions
04:00 Manual definition of compute graph
06:23 A (primal) function library
07:42 Primal forward execution of the compute graph
13:38 Library of primitive with pullbacks
19:40 Implementing reverse-mode AD via a vJp transformation
27:50 Testing the AD
29:07 Syntactic Sugar
30:39 Outro

Auf dieser Seite können Sie das Online-Video Reverse Mode Autodiff in Python (general compute graph) mit der Dauer stunde minuten sekunde in guter Qualität ansehen, das der Benutzer Machine Learning & Simulation 10 Juni 2024 hochgeladen hat, den Link mit Freunden und Bekannten teilen, dieses Video wurde auf Youtube bereits 2,031 Mal angesehen und es wurde von 93 den Zuschauern gefallen. Viel Spaß beim Betrachtenden Zuschauern gefallen!