Xcom Exclusive | Airflow

Inside your DAG, push with a unique key per execution date:

def push_exclusive(ti):
    ti.xcom_push(key=f"run_ti.execution_date_data", value="sensitive")
def pull_exclusive(ti):
# Only allowed to pull its own execution date's key
key = f"run_ti.execution_date_data"
return ti.xcom_pull(task_ids="push_exclusive", key=key)
 airflow xcom exclusive

Scenario:

Implement with Redis:

Add reliability:

This is the most critical constraint. Because XComs live in the metadata database, they are not designed for large datasets. Inside your DAG, push with a unique key