How to Solve Python Memory Problems with SQLite UDFs

Published: 31 January 2019
on channel: Software for Science
1,541
34

Have you ever run into MemoryError in Python? How can SQLite help you with large datasets?

In this video, we will show you how to use SQLite and write User Defined Functions (UDF) to solve resource problems when you deal with data.

1:23 Why SQLite?
3:24 Pass data between Python and SQLite
5:09 Why not bring it into memory?
7:29 What is serialization?
8:59 Explanation of code
11:09 json_each, and SQLite built-ins
13:42 User Defined Functions (UDF) examples
15:39 Why not memory? Revisited.
16:57 *Turning Python function into SQLite function*
18:25 How long does a UDF live?
19:37 Why you should remember to close the connection.

CODE: https://github.com/SeanTater/software...

MUSIC
Nimbus by Eveningland https://www.youtube.com/audiolibrary/...

SPECIAL THANKS
Dave Harned from Junior Developer Toolbox https://juniordevelopertoolbox.com/
Stephen Welch from Welch Labs
   / @welchlabs  
Ben Drozdoff


On this page of the site you can watch the video online How to Solve Python Memory Problems with SQLite UDFs with a duration of hours minute second in good quality, which was uploaded by the user Software for Science 31 January 2019, share the link with friends and acquaintances, this video has already been watched 1,541 times on youtube and it was liked by 34 viewers. Enjoy your viewing!