Podcast profile
By Astronomer
Welcome to The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI— the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward. Join us each week, as we explore the current state, future and potential of Airflow with leading thinkers in the community, and discover how best to leverage this workflow management system to meet the ever-evolving needs of data engineering and AI ecosystems. Podcast Webpage: https://www.astronomer.io/podcast/
Latest episode
28 min
Classification review pending
Topic, format, and audience-level labels are published only after a manual source review. This show remains fully discoverable by its researched technology sectors.
Listen from the original source
This directory never re-hosts or republishes podcast audio.
Official sources checked
Episode audio and show facts come from the checked-in refresh of the show’s public sources.
From the feed
When a real-time loan eligibility scoring pipeline is built on cron jobs, midnight pages are inevitable. In this episode, [Chandan Gowda](linkedin.com/in/chandan-gowda-a-h-744908195), Data Engineer at [Accion Labs](accionlabs.com), joins Kenten to discuss how his team uses Airflow across client projects, including a fi
Migrating two decades of legacy job orchestration is the kind of project most teams quietly avoid. In this episode, Marc Lamberti is joined by [Sanket Patel](linkedin.com/in/sanket-patel-a2a57b3a), Director of Engineering at Trading Technologies, to talk through how his team is consolidating 20 years of C#/.NET-based s
How do you orchestrate AI workflows when LLM outputs are non-deterministic and evaluation costs can quietly exceed compute costs? Shawn Feng, Head of Data at [Firework](firework.com), joins the show to walk through how his team uses Airflow to coordinate ingestion, embeddings, and evaluation pipelines for AI features i
Saks Global runs one of the largest retail data operations in the US, with around 8 million SKUs flowing across point-of-sale, e-commerce, catalog, and fraud detection systems into Snowflake. In this episode, [Shailesh Kadam](linkedin.com), Architect at [Saks Global](saks.com), joins Kenten to walk through how Airflow
Airflow 3.3 is here, with a set of features to help with the messy realities of production pipelines: persisting state across retries, reacting intelligently to different failure types, and partitioning assets by more than just time. In this episode, Marc Lamberti, Education Content Lead at [Astronomer](astronomer.io),
Running Apache Airflow at a major pension fund means balancing strict compliance requirements with the need to move fast on new capabilities. On this episode, Kowsy Narayan, Cloud Data Platform Lead, Data Engineering at [Ontario Teachers' Pension Plan](otpp.com), joins host Kenten Danas to walk through OTPP's cloud mig
Skimlinks runs a reporting platform that serves around 2,000 weekly publisher users, and the data infrastructure behind it runs on Airflow. In this episode, Julian Larralde, Director of Data Engineering at Skimlinks, walks through the stack, the migration from external task sensors to event-driven Assets, and a YAML-ba
JLR is the UK's largest automotive manufacturer, behind brands like Range Rover, Jaguar, Defender, and Discovery. In this episode, Najeeb Sulaiman, Senior Data Engineer at JLR, walks through how Airflow orchestrates data across manufacturing, supply chain, and finance — including a custom Tableau provider his team buil
Running Airflow at the scale of a national retailer means more than just scheduling. It means giving non-engineers a path to ship DAGs, and classifying thousands of runs to know which ones need attention. In this episode, Mateus Ferreira, Senior Data Engineer at Luiza Labs (the technology arm of Magazine Luiza, one of
In the data engineering world, the difference between a pipeline that works and one that's truly production-ready often comes down to a handful of deliberate decisions. William Orgertrice III, Data Engineer at Cargill, joins us to share the DAG design and monitoring practices he presented at Airflow Summit 2025 and how
In this episode, we take a step back from implementation-specific topics to explore what it actually takes to build a career in data engineering — and how AI is reshaping that path. Shrividya Hegde, a data and AI engineer and an Airflow champion in Astronomer’s Champions program, joins us to discuss getting into data e
We explore how a third-party logistics platform built its entire data orchestration layer on Airflow, and what that makes possible for developer teams and merchant-facing products alike. Filip Kunčar, Platform Director at ShipMonk Product Development, discusses migrating from a closed source tool to Airflow, orchestrat
Buğra Öztürk, Senior Data Engineer at Mollie and Committer and PMC member on the Apache Airflow project, joins us to walk through Airflow CTL — what it is, how it differs from the existing Airflow CLI and where it is headed under AIP-94. Key Takeaways: 00:00 Introduction. 03:10 Buğra has contributed to Airflow since 20
In this episode, we explore the newly released Apache Airflow common AI provider — what problem it solves, how it was built and what's coming next. Kaxil Naik, Senior Director of Engineering at Astronomer and Apache Airflow PMC member, and Pavan Kumar Gopidesu, Lead Data Engineer at Experian and Apache Airflow PMC memb
Aviation data pipelines run on strict 28-day publication cycles, and the margin for error is zero. In this episode, we're joined by Samantha Blaney Cuevas, Software Engineer at Jeppesen ForeFlight, to explore how her team orchestrates a complex, time-sensitive data pipeline with Airflow and where AI is starting to fit
We introduce Airflow 3.2 and its updates for teams that build and operate data pipelines. Astronomer’s Head of Customer Education, Marc Lamberti, and Senior Manager of Developer Relations, Kenten Danas, break down what’s new, from asset partitioning to Async Python tasks and DAG versioning. They explore how these updat
Lessons from the past decade of data engineering reveal how much the ecosystem has changed and what has stayed surprisingly consistent. In this episode, Benjamin Rogojan, Owner and Data Consultant at Seattle Data Guy, joins us to reflect on how the data engineering landscape has evolved alongside Apache Airflow. We exp
Data quality is not optional when you manage credit data at scale. In this episode, Ashir Alam, Senior Data Engineer at Credit Karma, joins us to share how his team acts as the gatekeeper for credit data ingestion, how they standardize data quality with Airflow and DAG Factory and how they scale safely across thousands
Modern Airflow isn’t just orchestration. It's a contribution. In this episode, we explore how open source investment drives real performance gains and deeper observability. We’re joined by Christos Bisias, Open Source Software Engineer, Apache Airflow at G-Research, to discuss how his team uses Airflow for large-scale
In this episode, Karan Alang, Principal Software Engineer at Versa Networks, joins the conversation to discuss how Airflow can be used to automate threat intelligence in modern cybersecurity environments. He explains the growing scale of cloud computing, the profitability of hacking and the shortage of SOC analysts. Ka
In this episode, we explore how teams scale Apache Airflow in complex environments and what it takes to make orchestration work across many stakeholders. We look at real-world challenges around visibility, ownership and predictability as data platforms grow. Egor Tarasenko, Data and AI Engineer at Ponder Labs, joins us
Modern data orchestration at scale demands reliability, speed and thoughtful adoption of new tooling. As organizations grow, keeping pipelines efficient while supporting more teams becomes a critical challenge. In this episode, we’re joined by Ethan Shalev, Data Engineer at Wix, to discuss how Wix operates Airflow at m
Strong data orchestration is as much about culture and visibility as it is about technology. As data platforms scale, teams need systems that reduce cognitive load while increasing reliability and observability. In this episode, Carlos Daniel Puerto Niño, Senior Analytics Engineer and Data Analyst at Addi, joins us to
Real-time data expectations are reshaping how modern data teams think about orchestration and dependencies. As event-driven architectures become more common, teams need to rethink how pipelines react to data changes, rather than schedules. In this episode, Andrea Bombino, Co-Founder and Head of Analytics Engineering at
A strong data-driven mindset underpins how fintech teams scale analytics, infrastructure and decision-making across the business. In this episode, Jaime Oliveira, Lead Data Engineer at Uphold, joins us to discuss how Uphold structures its data organization and orchestration strategy. Jaime shares how the team uses Airf