Functional Programming in Python: Parallel Processing with "concurrent.futures"
https://dbader.org/python-tricks ► Master Python's advanced features and write faster + cleaner code
In this Python tutorial you'll learn how to do multithreading and parallel programming in Python using functional programming principles and the "concurrent.futures" module.
We'll take the example data set based on an immutable data structure that we previously transformed using the built-in "map" function. But this time we'll process the data it in parallel, across multiple threads using Python 3's "concurrent.futures" module available in the standard library.
You'll see step by step how to parallelize an existing piece of Python code so that it can execute much faster and leverage all of your available CPU cores and computing power.
You'll learn how to use the "ProcessPoolExecutor" and "ThreadPoolExecutor" classes and their parallel "map" implementations that makes parallelizing most Python code written in a functional style a breeze.
By knowing the difference between both executors available in the concurrent.futures module you'll be able to parallelize your Python functions across multiple threads and across multiple processes.
I'll also give you a brief introduction to the Python "Global Interpreter Lock", also known as the "GIL", and how you can work around its limitations by using the correct executor implementation.
Once again we'll use our little testbed program from the last video to measure the execution time with the "time.time()" function. This allows us to compare the single-threaded and multithreaded implementations of the same algorithm.
FREE COURSE – "5 Thoughts on Mastering Python" https://dbader.org/python-mastery
SUBSCRIBE TO THIS CHANNEL: https://dbader.org/youtube
* * *
► Python Developer MUGS, T-SHIRTS & MORE: https://nerdlettering.com
► PythonistaCafe – A peer-to-peer learning community for Python developers: https://www.pythonistacafe.com
FREE Python Coding Tutorials & News:
» Python Tutorials: https://dbader.org
» Python News on Twitter: / @dbader_org
» Weekly Tips for Pythonistas: https://dbader.org/newsletter
» Subscribe to this channel: https://dbader.org/youtube