About this role
Lead Data Engineer responsible for designing and delivering scalable data platforms, including data lakes, event streaming, and data fabric solutions, with a focus on governance and AI readiness in healthcare.
Key Responsibilities
- Lead design and development of data platforms
- Drive data governance and lineage
- Oversee data mesh and centralized models
- Build metadata management and data quality processes
- Guide CI/CD pipelines and testing frameworks
Technical Overview
Hands-on experience with data engineering for large-scale cloud environments, including Python, Azure Databricks, Kafka-based streaming, and containerized deployments (Docker/Kubernetes). Strong data governance, lineage, and QA practices are expected.
Ideal Candidate
The ideal candidate is a senior data engineer with 12+ years of experience designing and building data platforms, strong expertise with data lakes, event streaming, and data fabric, and hands-on work with cloud solutions (AWS/Azure/GCP). They should lead teams, own data governance and lineage, and contribute to AI/ML initiatives in a healthcare context.
Must-Have Skills
12 + years of Data engineering experience with Hands-on experience with Data LakesEvent StreamingData FabricDevOps8 + years of experience designing software applications using best practices and design-first approach8 + years of experience with PythonAzure DatabricksPysharkSnowflakeExperience in event-based streaming using KafkaExperience designing and developing cloud solutions using AWSAzure or GoogleExperience leading engineering teams in end-to-end deliveryFederated data architectures (Data Mesh) and centralized modelsMetadata managementdata lineagegovernancediscoveryand qualityExperience with Oracle or SQL ServerGitHub / GitLabSonarQubeJUnitTDD/BDDCI/CD pipelines and code quality toolsDocker and KubernetesAgile methodologiesExposure to AIReact or Angular
Nice-to-Have Skills
Familiarity with ML frameworksMLOpsMicroservices design and developmentExperience with AI
Tools & Platforms
GitHubGitLabSonarQubeJUnitSeleniumTDDBDD
Required Skills
Data LakesEvent StreamingData FabricDevOpsPythonAzure DatabricksPysharkSnowflakeKafkaJSONAvroAWSAzureGoogle CloudData Meshmetadata managementdata lineagegovernancediscoveryqualityOracleSQL ServerGitHubGitLabSonarQubeJUnitSeleniumTDDBDDCI/CD pipelinesDockerKubernetesAgileAIReactAngular
Hard Skills
Data LakesEvent StreamingData FabricDevOpsPythonAzure DatabricksPysharkSnowflakeKafkaJSONAvroAWSAzureGoogle CloudData Meshmetadata managementdata lineagegovernancediscoveryqualityOracleSQL ServerGitHubGitLabSonarQubeJUnitSeleniumTDDBDDCI/CD pipelinesDockerKubernetesAgileAIReactAngular
Soft Skills
team collaborationmentoringleadershipproblem-solvingcommunication
Keywords for Your Resume
Lead Data Engineerdata lakesevent streamingdata fabricDevOpsPythonAzure DatabricksPysharkSnowflakeKafkaJSONAvroAWSAzureGoogle CloudData Meshmetadata managementdata lineagegovernancediscoveryqualityOracleSQL ServerGitHubGitLabSonarQubeJUnitSeleniumTDDBDDCI/CD pipelinesDockerKubernetesAgileAIReactAngular
Deal Breakers
Lack of 12+ years data engineering experience, No experience with data lakes or event streaming, Inability to work in a hybrid remote/on-site model
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile