Course Description
Apache Airflow is a configuration-as-code OSS solution for workflow automation that is positioned as a replacement of cron-like scheduling systems. Written in Python, Airflow enables developers to programmatically author, schedule for execution, and monitor highly configurable complex workflows.
Data Pipelines and Orchestration with Apache Airflow Dec 2025Data Pipelines and Orchestration with Apache Airflow
A Traditional ETL Approach
Apache Airflow Defined
Airflow Core Components
The Component Collaboration Diagram
Workflow Building Blocks and Concepts
Airflow CLI
Main Configuration File
Extending Airflow
Jinja Templates
Variables and Macros
Summary
Chapter 2. Apache Airflow Web UIWeb UI - the Landing (DAGs) PageWeb UI - the DAG Graph ViewRun Status LegendsThe Pause Button (Trigger Latch)The DAG Triggering/Job Checking SequenceThe Control Panel for a TaskSample Log File Messages (Abridged for Space)Summary
Web UI - the Landing (DAGs) Page
Web UI - the DAG Graph View
Run Status Legends
The Pause Button (Trigger Latch)
The DAG Triggering/Job Checking Sequence
The Control Panel for a Task
Sample Log File Messages (Abridged for Space)
Summary
Chapter 3. Anatomy of a DAG and SchedulingWhat is a DAG?Scheduled and Manually Triggered DAG RunsThe DAG ObjectTasksTask LifecycleOperatorsIdempotent OperatorsOperator TypesAirflow Common OperatorsSpecifying DependenciesAssociating Operators with a DAGAssociating Operators Using the "With DAG" Statement ExampleAssociating Operators with DAG Using the Operator's ConstructorThe default_args ParameterPassing DAG Parameters Through Web UIDAG Run SchedulingExamples of the schedule_interval ParameterDAG Scheduling NuancesUnderstanding The Backfill ProcessKilling/Stopping DAG RunsAn XCom Messaging ExampleSummary
What is a DAG?
Scheduled and Manually Triggered DAG Runs
The DAG Object
Tasks
Task Lifecycle
Operators
Idempotent Operators
Operator Types
Airflow Common Operators
Specifying Dependencies
Associating Operators with a DAG
Associating Operators Using the "With DAG" Statement Example
Associating Operators with DAG Using the Operator's Constructor
The default_args Parameter
Passing DAG Parameters Through Web UI
DAG Run Scheduling
Examples of the schedule_interval Parameter
DAG Scheduling Nuances
Understanding The Backfill Process
Killing/Stopping DAG Runs
An XCom Messaging Example
Summary
Related Certificates
Earning a Saint Louis University Workforce Center certificate allows you to enhance your career options by networking with peers and establishing valuable industry contacts.
ISACA Professional Certificate
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud.
See Details
Artificial Intelligence and Large Language Models Foundations
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud.
See Details
Artificial Intelligence and Machine Learning for Cybersecurity Operations
Artificial intelligence (AI) and machine learning (ML) are revolutionizing the cybersecurity industry. The emergence of ML as a tool for...
See Details