Apache Airflow for Data Science - How to Work with Variables
Hardcoding values in your Airflow DAGs is a bad practice. Learn how to use Airflow variables …
10 Posts
Airflow
Apache Airflow for Data Science - How to Download Files from Amazon S3
Learn how to download files from Amazon S3 (AWS) to your local machine with Apache Airflow and …
Apache Airflow for Data Science - How to Upload Files to Amazon S3
Learn how to setup an Amazon S3 (AWS) Bucket and how to upload files from local disk with …
Apache Airflow for Data Science - How to Work with REST APIs
Learn to work with REST APIs in Apache Airflow by utilizing HttpSensor and HttpOperator Airflow …
Apache Airflow for Data Science - How to Communicate Between Tasks with Airflow XComs
Learn to send and receive data between Airflow tasks with XComs, and when you shouldn’t …
Apache Airflow for Data Science - How to Run Tasks in Parallel
Build a Data Pipeline (DAG) in Apache Airflow that makes four GET API requests in Parallel.
Apache Airflow for Data Science - How to Migrate Airflow Metadata DB to Postgres and Enable Parallel Execution
Apache Airflow doesn’t run tasks in parallel by default - but there’s an easy fix. …
Apache Airflow for Data Science - How to Work With Databases (Postgres)
Learn how to extract, transform, and load data with Airflow and Postgres database by coding a …
Apache Airflow for Data Science - How to Write Your First DAG in 10 Minutes
Apache Airflow is a common tool used by Data Engineers. Learn how to write your first data …
Apache Airflow for Data Science - How to Install Airflow Locally
Want to learn Apache Airflow as a Data Engineer? Start by installing it locally. Go from zero …
