Basic Data Pipeline Demonstration Using Apache Airflow (Python)
Basic Data Pipeline Demonstration Using Apache Airflow (Python)
Convert rows in CSV file to JSON, every 5 minutes (scheduled in Airflow)
File paths are according to Docker Container path.
Check by
1. Login to the Airflow container with docker exec -it [ContainerName] /bin/bash (or type it in cmd)
2. List the files in that directory with ls /opt/airflow/data/
In this case, I put my data.csv file in /opt/airflow/dags folder so I won't have to edit the yaml file.
On this page of the site you can watch the video online Basic Data Pipeline Demonstration Using Apache Airflow (Python) with a duration of hours minute second in good quality, which was uploaded by the user Pattarapon Buathong 11 March 2022, share the link with friends and acquaintances, this video has already been watched 70 times on youtube and it was liked by 0 viewers. Enjoy your viewing!
