Podcast profile
By Astronomer
Welcome to The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI— the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward.
Latest episode
20 min
Topics
Building and operating data pipelines, transformations, orchestration, and data platforms.
Data modeling, transformation, semantic layers, metrics, and analytics workflows.
Advanced manufacturing, production systems, factories, and industrial digitalization.
Formats
Detailed technical, architectural, or research-oriented examination.
Host-led conversation where a guest supplies most of the subject matter.
Audience level
Regularly assumes specialist knowledge and discusses implementation, architecture, protocols, code, or research in depth.
Stay connected
From the feed
Episodes shown — 10 of 25
Orchestrating data across more than 30 companies means most Airflow users on the platform aren't data engineers. In this episode, Kenten Danas talks with Lucas Trubiano, Data Engineer at itti, the technology company within Grupo Vázquez in Paraguay.
Manufacturing data pipelines can't afford silent failures. When quality decisions and shop-floor visibility depend on Airflow, a broken DAG at midnight can cost real money.
Strict sequential execution across DAGs sounds simple until you have scheduled pipelines and event-driven pipelines writing to the same MongoDB collections.
Financial data pipelines have to be right the first time. Key Takeaways: 00:00 Introduction. And be sure to subscribe so you never miss any of the insightful conversations.
Running a travel platform means dealing with fast-moving inventory, real-time fraud detection, and heavy performance marketing attribution, all while keeping data trustworthy for a data-driven org. Key Takeaways: 00:00 Introduction. And be sure to subscribe so you never miss any of the insightful conversations.
Airflow and Spark are one of the most common combinations in modern data platforms, but the orchestration decisions made on the Airflow side often determine whether Spark jobs run fast and cheap or slow and expensive. Key Takeaways: 00:00 Introduction.
When a real-time loan eligibility scoring pipeline is built on cron jobs, midnight pages are inevitable. Key Takeaways: 00:00 Introduction. And be sure to subscribe so you never miss any of the insightful conversations.
Migrating two decades of legacy job orchestration is the kind of project most teams quietly avoid. The conversation covers orchestrator selection, observability, the operational economics of managed Airflow, the human side of migrations, and where AI fits into the new data platform. Key Takeaways: 00:00 Introduction.
How do you orchestrate AI workflows when LLM outputs are non-deterministic and evaluation costs can quietly exceed compute costs? The conversation covers the technical stack, the challenges of validating LLM output, cost guardrails, and what Shawn wants to see next from the Airflow project.
Saks Global runs one of the largest retail data operations in the US, with around 8 million SKUs flowing across point-of-sale, e-commerce, catalog, and fraud detection systems into Snowflake. Key Takeaways: 00:00 Introduction.