Position Details

Type Not Specified

Experience mid

Exp. Years 3+ years

Education Not specified

Category Data & Analytics

About this role

Seeking a senior data engineer to design and build scalable data pipelines, manage master data, and ensure data quality and compliance across platforms.

Key Responsibilities

Build scalable data pipelines
Manage master data
Implement data quality checks
Ensure data governance and compliance
Lead data migration projects

Technical Overview

Environment includes PySpark, Delta Lake, Informatica MDM Hub, Informatica CDI, Data Governance tools, and data pipeline migration strategies.

Ideal Candidate

The ideal candidate is a mid-level data engineer with extensive experience in building scalable data pipelines using PySpark and Delta Lake, and managing master data with Informatica MDM Hub. They possess strong leadership skills and a solid understanding of data governance and compliance frameworks.

Must-Have Skills

Experience with PySpark and Delta LakeDesigning scalable data pipelinesExperience with Informatica MDM HubConfiguring match/merge rulesImplementing data quality checksData governance and complianceData pipeline migration

Nice-to-Have Skills

Informatica CDIData lineage managementCJIS compliance

Tools & Platforms

PySparkDelta LakeInformatica MDM HubInformatica CDIInformatica CDQDatabricks Unity Catalog

Required Skills

PySparkDelta LakeMedallion ArchitectureInformatica MDM HubData GovernanceUnity CatalogETL/ELTInformatica CDIData QualityData pipelinesData management

Hard Skills

PySparkDelta LakeMedallion ArchitectureInformatica MDM HubData GovernanceUnity CatalogETL/ELTInformatica CDIData QualityInformatica CDQData pipelinesData managementData complianceData lineage

Soft Skills

LeadershipMentoringTechnical collaborationProblem-solvingCommunication

Industry & Role

Industry Technology

Job Function Designing and managing enterprise data pipelines and master data systems

Keywords for Your Resume

PySparkDelta LakeMedallion ArchitectureInformatica MDM HubData GovernanceUnity CatalogETL/ELTInformatica CDIData QualityInformatica CDQData pipelinesData managementData complianceData lineage

Deal Breakers

Lack of experience with PySpark or Delta Lake, No familiarity with Informatica MDM Hub, Insufficient knowledge of data governance and compliance, No experience with data pipeline migration

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Senior Databricks Engineer

Get matched to jobs like this