How to Install Apache Airflow on Windows using Docker Container |

How to Install Apache Airflow on Windows using Docker Container

1. create a directory named airflow and inside that create three sub directories - logs, dags, plugins and place docker-compose.yaml file inside airflow directory
2. install docker desktop for windows
3. download docker-compose.yaml file of airflow latest release
https://airflow.apache.org/docs/apach...

Airflow releases: https://airflow.apache.org/docs/apach...

4. docker-compose up airflow-init
5. docker-compose up

Cleanup - below command will delete all the conainer and free up the volumes and memory
`Run docker-compose down --volumes --remove-orphans`
command in the directory you downloaded the docker-compose.yaml file

airflow-scheduler - The scheduler monitors all tasks and DAGs, then triggers the task instances once their dependencies are complete.
airflow-webserver - The webserver is available at http://localhost:8080.
airflow-worker - The worker that executes the tasks given by the scheduler.
airflow-triggerer - The triggerer runs an event loop for deferrable tasks.
airflow-init - The initialization service.
postgres - The database.
redis - The redis - broker that forwards messages from scheduler to worker.

Reference: https://airflow.apache.org/docs/apach...

For any queries, you can connect with me using below links:

LinkedIn:   / ashutoshtripathiai  
Instagram:   / ashutoshtripathi_ai  
Twitter:   / ashutosh_ai  
Website: https://ashutoshtripathi.com

If you want to message me directly, then connect with me on LinkedIn and send a direct message.

If you want to discuss any topic around ML or have queries related to job switch, please feel free to schedule a 30 min call with me, I am available on topmate: https://topmate.io/ashutosh_ai

#airflow #mlops #ashutosh_ai