Position Details
About this role
This role is a senior data engineering position focused on building and scaling high-throughput streaming pipelines and real estate datasets. You will lead improvements in data quality and observability and contribute to AI-driven tooling that helps triage and resolve data quality issues.
Key Responsibilities
- Build and scale high-throughput streaming pipelines
- Model and deliver production-grade real estate datasets
- Strengthen data quality and observability with monitoring and alerting
- Leverage AI tooling to triage and debug data quality issues
- Drive platform architecture for AI-powered product delivery
Technical Overview
The stack explicitly includes Airflow, Spark Streaming, Kafka, and Iceberg for ingesting and operating streaming pipelines. The role also covers dataset modeling and transformation logic, plus data quality checks, monitoring, and alerting to maintain freshness, accuracy, and cost balance.
Ideal Candidate
The ideal candidate is a senior data engineer who has already built with AI daily and uses Claude Code as a core workflow. They have strong experience designing and operating high-throughput streaming pipelines using Airflow, Spark Streaming, Kafka, and Iceberg, and they improve data quality with checks, monitoring, and alerting.
Must-Have Skills
Tools & Platforms
Required Skills
Hard Skills
Soft Skills
Industry & Role
Keywords for Your Resume
Deal Breakers
Must have hands-on experience with Airflow, Spark Streaming, Kafka, and Iceberg, Must demonstrate experience implementing data quality checks, monitoring, and alerting
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile