Building Data Pipelines Using Python Tutorial | Data Pipelines Using Python Course

Published: 06 May 2023
on channel: Alpha Brains Courses

9,055

212

About This Course
This Python 3 course teaches data professionals how to create data pipelines and automate workflows using various frameworks like Airflow and Luigi. Assumes intermediate-level Python experience and covers important workflow concepts. Gain practical experience in parallelizing data analysis and writing data tests using property-based testing.

If you are involved in ANY form of online collaboration, then have I got the all-in-one solution for you! Try it here (it's completely free) - https://www.taskade.com/?via=taskade Check out Sam Beckam's video for a detailed video: • Taskade: An INCREDIBLE Collaboration ...

```
Show your Love & Support:
/ alphabrains
☕️ [Buy me a coffee!] (https://www.buymeacoffee.com/alphabrains)
Would Help me make more content like this one :)

```

Timestamp

Introduction
0:00:00 Welcome To The Course

Automation 101
0:02:23 Introduction To Automation
0:05:11 Adventures With Servers
0:11:48 Being A Good Systems Caretaker
0:17:51 What Is A Queue?
0:20:23 What Is A Consumer? What Is A Producer?

Easy Task Processing With Celery
0:22:23 Why Celery?
0:24:12 Celery Architecture & Set Up
0:29:37 Writing Your First Tasks
0:37:26 Deploying Your Tasks
0:43:34 Scaling Your Workers
0:52:26 Monitoring With Flower
0:57:31 Advanced Celery Features****

Scaling Data Analysis With Dask
1:03:31 Why Dask?
1:06:32 First Steps With Dask
1:16:40 Dask Bags
1:26:58 Dask Distributed

Data Pipelines With Luigi & Airflow
1:36:56 Python Data Pipelines
1:37:06 What Are Data Pipelines? What Is Dag?
1:39:43 Luigi And Airflow: A Comparison
1:45:33 First Steps With Luigi
1:52:45 More Complex Luigi Tasks
2:02:02 Introduction To Hadoop
2:10:23 First Steps With Airflow
2:18:30 Custom Tasks With Airflow
2:27:46 Advanced Airflow: Subdags And Branches
2:39:03 Using Luigi With Hadoop

Other Workflow Frameworks
2:49:18 Apache Spark
2:57:46 Apache Spark Streaming
3:04:18 Django Channels
3:13:57 And Many More

Testing With Pipelines
3:19:56 Testing pipelines
3:20:06 Introduction To Testing With Python
3:27:30 Property-Based Testing With Hypothesis

Conclusion
3:33:39 What's Next?

SEO
#howtobuilddatapipelines ,#BuildingDataPipelinesUsingPythonTutorial , #BuildingDataPipelinesWithPython

On this page of the site you can watch the video online Building Data Pipelines Using Python Tutorial | Data Pipelines Using Python Course with a duration of hours minute second in good quality, which was uploaded by the user Alpha Brains Courses 06 May 2023, share the link with friends and acquaintances, this video has already been watched 9,055 times on youtube and it was liked by 212 viewers. Enjoy your viewing!

158