About this role
This role involves leading data engineering initiatives, building scalable data pipelines, and implementing advanced analytics solutions in a cloud environment, collaborating with cross-functional teams.
Key Responsibilities
- Build and optimize data pipelines
- Develop APIs and microservices
- Implement data quality and monitoring
- Lead technical solutions
- Collaborate with stakeholders
Technical Overview
The position requires expertise in Python, AWS, data architecture, ETL processes, microservices, APIs, and modern data tools like Snowflake, Kafka, and SageMaker, with a focus on data quality and governance.
Ideal Candidate
The ideal candidate is a senior data engineer with over 7 years of experience in building and maintaining data pipelines, expertise in Python, cloud platforms like AWS, and leadership skills to mentor teams. They are well-versed in modern data architectures, data governance, and advanced analytics tools.
Must-Have Skills
7+ years experience as Data Engineer or Software EngineerStrong Python programmingExperience with data pipelines and architectureExperience with cloud platforms (AWS or equivalent)Experience with data APIs and microservicesExperience with data quality and monitoringExperience with orchestration tools (Airflow)
Nice-to-Have Skills
Data mesh conceptsRecommender systemsFraud detectionKnowledge graphsVector databasesSnowflakePostgresDynamoDBKafkaFivetrandbtSageMakerData observability toolsData governance
Tools & Platforms
PythonAWSSnowflakePostgresDynamoDBKafkaFivetrandbtAirflowSageMakerDatadogPagerDutyDockerKubernetes
Required Skills
Data EngineeringETL processesPythonData pipelinesData modelingAPIsMicroservicesAWSCloud platformsGitCI/CDDockerKubernetesSnowflakePostgresDynamoDBKafkaFivetrandbtAirflowData observabilityData governanceData meshRecommender systemsFraud detectionPersonalizationMarketing scienceVector databasesKnowledge graphsSageMakerDatadogPagerDuty
Hard Skills
Data EngineeringETL processesPythonData pipelinesData modelingAPIsMicroservicesAWSCloud platformsGitCI/CDDockerKubernetesSnowflakePostgresDynamoDBKafkaFivetrandbtAirflowData observabilityData governanceData meshRecommender systemsFraud detectionPersonalizationMarketing scienceVector databasesKnowledge graphsSageMakerDatadogPagerDuty
Soft Skills
LeadershipCommunicationMentoringCollaborationProblem-solvingContinuous learning
Keywords for Your Resume
Data EngineerData PipelinesETL processesPythonAWSCloud platformsAPIsMicroservicesData modelingData pipelinesData qualityData monitoringAirflowData meshRecommender systemsFraud detectionKnowledge graphsVector databasesSageMakerData observabilityData governance
Deal Breakers
Less than 7 years of experience in data engineering or software engineering, Lack of Python programming skills, No experience with cloud platforms (AWS or equivalent), No experience with data APIs or microservices, Lack of leadership or mentoring experience
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile