Functional Programming in Python: Parallel Processing with "multiprocessing"

Published: 04 October 2017
on channel: Real Python
59,856
1.1k

https://dbader.org/python-tricks ► Write better & cleaner code using Python's advanced features

In this tutorial you'll learn how to do parallel programming in Python using functional programming principles and the "multiprocessing" module.

We'll take the example data set based on an immutable data structure that we previously transformed using the built-in "map" function. But this time we'll process the data it in parallel, across multiple CPU cores using the Python "multiprocessing" module available in the standard library.

You'll see step by step how to parallelize an existing piece of Python code so that it can execute much faster and leverage all of your available CPU cores.

You'll learn how to use the "multiprocessing.Pool" class and it's parallel "map" implementation that makes parallelizing most Python code that's written in a functional style a breeze.

As part of this video we'll build a little testbed program first that we can use to measure the execution time with the "time.time()" function, so that we can compare the single-threaded and multithreaded implementations of the same algorithm.

Stay tuned for the next video in the series where you'll learn how to make your Python programs multithreaded using the "concurrent.futures" module as an alternative way to implement concurrency.

FREE COURSE – "5 Thoughts on Mastering Python" https://dbader.org/python-mastery

SUBSCRIBE TO THIS CHANNEL: https://dbader.org/youtube

* *

► Python Developer MUGS, T-SHIRTS & MORE: https://nerdlettering.com

► PythonistaCafe – A peer-to-peer learning community for Python developers: https://www.pythonistacafe.com

FREE Python Coding Tutorials & News:
» Python Tutorials: https://dbader.org
» Python News on Twitter:   / @dbader_org  
» Weekly Tips for Pythonistas: https://dbader.org/newsletter
» Subscribe to this channel: https://dbader.org/youtube


On this page of the site you can watch the video online Functional Programming in Python: Parallel Processing with "multiprocessing" with a duration of hours minute second in good quality, which was uploaded by the user Real Python 04 October 2017, share the link with friends and acquaintances, this video has already been watched 59,856 times on youtube and it was liked by 1.1 thousand viewers. Enjoy your viewing!