GPTQ Quantization EXPLAINED
If you need help with anything quantization or ML related (e.g. debugging code) feel free to book a 30 minute consultation session! https://calendly.com/oscar-savolainen
I'm also available for long-term freelance work, e.g. for training / productionizing models, teaching AI concepts, etc.
Video Summary:
In this video we go over the GPTQ neural network quantization paper, and explain the whole thing. We explain the theory behind the use of the Hessian, the derived equations, the GPU optimizations, and the use of Cholesky decomposition.
Links:
GPTQ paper: https://arxiv.org/abs/2210.17323
Intro to quantization: • Understanding int8 neural network quantiza... Case of "power limited" GPUs: https://www.thonking.ai/p/strangely-m...
Video animations made with the help of the https://github.com/3b1b/manim library
Timestamps:
00:00 Intro
01:43 Motivation: why invent GPTQ in the first place?
05:47 History of papers
07:00 Basic idea of GPTQ
08:10 Small mistake in the GPTQ paper
10:27 Explanation of Hessian matrix
12:10 Talyor Series
15:10 How to derive the pick-row and update-weights equations
18:37 Gaussian elimination
19:57 Original contributions of GPTQ - computational optimization
23:12 Computational bottlenecks and lazy batch updates
29:00 Cholesky decomposition
31:58 Conclusion
On this page of the site you can watch the video online GPTQ Quantization EXPLAINED with a duration of hours minute second in good quality, which was uploaded by the user Oscar Savolainen 27 October 2024, share the link with friends and acquaintances, this video has already been watched 4,025 times on youtube and it was liked by 145 viewers. Enjoy your viewing!
