| .astro | ||
| alembic | ||
| configmaps_files | ||
| dags | ||
| image | ||
| local_lib | ||
| plugins | ||
| scripts | ||
| tests/dags | ||
| .dockerignore | ||
| .gitignore | ||
| _git_init.sh | ||
| airflow_settings.yaml | ||
| alembic.ini | ||
| docker-compose.override.yml | ||
| docker-compose.override.yml-llm | ||
| docker-compose.override.yml.KO | ||
| docker-compose.override.yml.minio | ||
| docker-compose.override.yml.OK | ||
| docker-compose.override.yml.XXX | ||
| Dockerfile | ||
| Makefile | ||
| Makefile.old | ||
| output.dot_json | ||
| packages.txt | ||
| poetry.lock | ||
| pyproject.toml | ||
| pyproject.toml.k | ||
| pyproject.toml.x | ||
| README.md | ||
| requirements.txt | ||
| TIPS.md | ||
| Untitled | ||
airflow-dag
S3 XCOM
https://www.astronomer.io/docs/learn/xcom-backend-tutorial
Listener
https://www.astronomer.io/docs/learn/airflow-listeners
Kafka
https://www.astronomer.io/docs/learn/airflow-kafka/
Exemple
https://github.com/matsudan/airflow-dag-examples
Deferable operator
https://medium.com/apache-airflow/understanding-deferable-operators-in-apache-airflow-8b32c1615038 https://github.com/AlvaroCavalcante/airflow-custom-deferrable-dataflow-operator
Parallel dags
Airflow 3 SDK -- httpx
curl -sSL install.astronomer.io | sudo bash -s astro dev init astro dev start
Fast API - generic API
https://medium.com/@jkishan421/building-dynamic-api-responses-with-generics-in-fastapi-972fa1f52d54
gist env decorator / context
Cool repos
Airflow async / polar / fernet (@task.load_fernet)...
https://github.com/phi-friday?tab=repositories
plugin
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/plugins.html
XXX download graph
http://localhost:8080/ui/ /structure_data?dag_id=dag_test_async airflow dags show dag_test_async > /tmp.dot