Dask DataFrame: An Introduction

Published: 23 September 2019
on channel: Dask

52,128

383

In this video, Matt Rocklin gives a brief introduction to Dask DataFrames.

Dask is a free and open-source library for parallel computing in Python. Dask is a community project maintained by developers and organizations.

Dask helps you scale your data science and machine learning workflows and also makes it easy to work with Numpy, pandas, and Scikit-Learn. Dask is a framework to build distributed applications that has been used with dozens of other systems like XGBoost, PyTorch, Prefect, Airflow, RAPIDS, and more.

Dask DataFrames scale pandas workflows, enabling applications in time series, business intelligence, and general data munging on big data. A Dask DataFrame is a large parallel DataFrame composed of many smaller Pandas DataFrames, split along the index. These Pandas DataFrames may live on disk for larger-than-memory computing on a single machine, or on many different machines in a cluster. One Dask DataFrame operation triggers many operations on the constituent Pandas DataFrames. Dask Dataframes coordinate many Pandas dataframes, partitioned along with an index.

Dask DataFrames coordinate many Pandas DataFrames/Series arranged along the index. A Dask DataFrame is partitioned row-wise, grouping rows by index value for efficiency. These Pandas objects may live on disk or on other machines. Because the dask.dataframe application programming interface (API) is a subset of the Pandas API, it should be familiar to Pandas users.

Share your feedback with us in the comments and let us know:

Did you find the video helpful?
Have you used Dask before?

Learn more at https://docs.dask.org/en/latest/dataf...

KEY MOMENTS
00:00 - Intro
00:15 - Start with Pandas
01:22 - Dask DataFrames
02:26 - Multiple files
03:14 - Dask DataFrame Partitions
04:33 - Mapping a Function Across All Partitions
06:35 - Metadata
06:46 - Parquet

On this page of the site you can watch the video online Dask DataFrame: An Introduction with a duration of hours minute second in good quality, which was uploaded by the user Dask 23 September 2019, share the link with friends and acquaintances, this video has already been watched 52,128 times on youtube and it was liked by 383 viewers. Enjoy your viewing!

13,324

276