Position Details

Salary $269K – $307K USD / year

Type Full-Time

Experience lead

Exp. Years 9+ years

Education Bachelor's Degree

Category Data & Analytics

About this role

Lead enterprise-scale data pipelines and platform strategy as a Director-level IC. Drive technology adoption, architecture, and engineering excellence across the data ecosystem, partnering with stakeholders to deliver scalable data solutions.

Key Responsibilities

Build awareness and drive adoption of modern technologies across the enterprise
Communicate with key stakeholders across the organization
Promote a culture of engineering excellence and reusable solutions
Shape use cases and implementation in a unified manner
Lead the way in creating next-generation talent and mentoring internal/external engineers

Technical Overview

Hands-on development with Python/SQL/Java/Scala and cloud data technologies (AWS, Snowflake, Databricks, Delta Lake). Deep expertise in Kafka, Flink, Spark, Airflow, and lakehouse architectures to evolve and mature enterprise-scale systems.

Ideal Candidate

The ideal candidate is a senior, director-level data engineering leader with 9+ years of data engineering and 5+ years in data architecture, plus deep AWS and enterprise data platform experience. Hands-on in Python/SQL/Java/Scala and modern big data tools (Kafka, Flink, Spark, Airflow, Snowflake, Databricks, Delta Lake, Iceberg) with a proven ability to mentor teams and drive enterprise data strategy.

Must-Have Skills

Bachelor's DegreeAt least 9 years of experience in data engineeringAt least 5 years of experience in data architectureAt least 3 years of experience building applications in AWSAt least 3 years of PythonScalaJavaor SQLAt least 3 years of Flink or SparkAt least 3 years of KafkaAt least 3 years of Delta Lake or IcebergAt least 3 years of building highly performanthighly resilient Big Data solutions

Nice-to-Have Skills

Masters' Degree10+ years of experience in data engineering3+ years of data modeling experience2+ years of experience with ontology standards for defining a domain3+ years of experience deploying machine learning models1+ year of experience deploying Agentic AI solutions3+ years of experience implementing big data processing solutions on AWS or other cloud providers3+ years of Snowflake or Databricks3+ years of Data Observability tooling (lineagedata qualitymonitoring or alerting)3+ years of detecting and remediating

Tools & Platforms

Amazon Web ServicesAWSApache AirflowAirflowDelta LakeIcebergSnowflakeDatabricksKafkaFlinkSpark

Required Skills

Distinguished Data EngineerAWSAmazon Web ServicesKafkaFlinkApache FlinkSparkApache SparkAirflowApache AirflowSnowflakeDatabricksDelta LakeIcebergPythonSQLJavaScalaData PipelinesData EngineeringData ArchitectureLakehouseBig Data

Hard Skills

PythonSQLJavaScalaAmazon Web ServicesAWSKafkaApache KafkaFlinkApache FlinkSparkApache SparkAirflowApache AirflowSnowflakeDatabricksDelta LakeIcebergLakehouse

Soft Skills

leadershipmentoringcommunicationstakeholder managementinfluencingcollaboration

Industry & Role

Industry Banking

Job Function Lead enterprise-scale data pipelines and platform strategy as a Director-level IC.

Clearance & Visa

Visa Sponsorship Yes

Keywords for Your Resume

Distinguished Data EngineerAWSAmazon Web ServicesKafkaFlinkApache FlinkSparkApache SparkAirflowApache AirflowSnowflakeDatabricksDelta LakeIcebergPythonSQLJavaScalaData PipelinesLakehouse

Deal Breakers

Less than 9 years of data engineering experience, No AWS experience, No Flink or Spark experience, No Kafka experience, No Delta Lake or Iceberg experience, No Bachelor's degree

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile