About this role
Manage and lead data engineering work focused on building and maintaining pipelines that extract, transform, validate, and deliver enterprise data. Optimize pipeline performance and ensure data quality, integrity, and consistency across systems.
Key Responsibilities
- Design and develop data pipelines
- Transform and cleanse data (aggregation, filtering, enrichment)
- Implement data validation and data quality checks within pipelines
- Optimize performance for scalability and efficiency
- Monitor systems, resolve bottlenecks, and tune query performance
Technical Overview
The role covers end-to-end pipeline engineering: extracting from databases, data warehouses, and APIs; transforming via cleansing, aggregation, filtering, and enrichment; and loading into appropriate storage systems. You will embed validation and data quality checks in pipelines, then monitor and tune systems using caching and indexing strategies to improve query performance.
Ideal Candidate
The ideal candidate is a data engineering leader who can design and operate enterprise data pipelines for reliable integration and transformation. They have strong hands-on experience ensuring data validation and data quality checks within pipelines, and they can optimize performance through caching, indexing, and monitoring while communicating effectively with senior stakeholders.
Must-Have Skills
designingdevelopingand maintaining data pipelinesbuilding data pipelines for data integrationtransformationand deliveryperforming data validation and cleaningimplementing data quality checks and validations within data pipelinesoptimizing data pipelines for performancescalabilityand efficiencymonitoring and tuning data systems and resolving performance bottlenecks
Tools & Platforms
APIsdata warehousesdatabases
Required Skills
data pipelinesdata integrationdata transformationdata validationdata cleaningdata quality checks and validationsperformance optimizationscalabilitymonitoringcachingindexingquery performance tuningdata consistencydata integritydata cleansingdata aggregationdata filteringdata enrichment
Hard Skills
data pipelinesdata integrationdata transformationdata storage systemsdata extractiondata loading (ETL/ELT implied)data validationdata cleaningdata consistencydata integritydata cleansingdata aggregationdata filteringdata enrichmentperformance optimizationscalabilitycaching strategiesindexing strategiesquery performance tuningdata quality checks and validationsmonitoring data systemsresolving performance bottlenecksenterprise information assets exploitation
Soft Skills
communicate clearly and concisely with senior leadershipadapt communication style for technical and non-technical audiencestranslate complex technical information into straightforward languagecritical thinkingcreative thinkingresearch and analysistrend recognitiondecision making with short and long-term implicationsadvising key decision makersinfluencing others using persuasion and negotiation tacticstake initiative
Keywords for Your Resume
Data Engineer ManagerData Engineer – ManagerData Engineerdata pipelinesdata integrationdata transformationdata extractiondata loadingdata storage systemsdata validationdata cleaningdata consistencydata integritydata cleansingdata aggregationdata filteringdata enrichmentperformancescalabilitymonitoringperformance bottleneckscachingindexingquery performancedata quality checksdata governancedata securitydata quality
Deal Breakers
Must be able to design, develop, and maintain data pipelines, Must have experience implementing data validation and data quality checks within pipelines
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile