Position Details

Type Contract

Experience mid

Exp. Years 5+ years

Education Not specified

Category Data & Analytics

About this role

Senior Data Engineer needed to design and implement scalable data pipelines using AWS and big data tools to support analytics and ML use cases. This remote, contract role requires hands-on expertise with Spark/Hadoop, Kafka, ETL/ELT, and data warehousing.

Key Responsibilities

Build scalable data pipelines for analytics and ML use cases
Enable ML use cases by prepared data pipelines
Work with cloud-based data platforms on AWS
ETL/ELT development and orchestration
Collaborate with data scientists and engineers

Technical Overview

Stack includes Python, SQL, AWS services (S3, Glue, Redshift, Lambda, EMR), Spark, Hadoop, Kafka; ETL/ELT orchestration and data lake/warehouse architectures; Airflow and CI/CD practices.

Ideal Candidate

The ideal candidate is a senior data engineer with 5+ years of data engineering experience, strong Python and SQL skills, and hands-on AWS experience (S3/Glue/Redshift/Lambda/EMR). They should design and optimize data pipelines for analytics and ML use cases and collaborate with data scientists and engineers in a consulting setting.

Must-Have Skills

5+ years of Data Engineering experienceStrong programming in Python and SQLHands-on experience with AWS services (S3GlueRedshiftLambdaEMR)Experience with big data tools like Apache SparkHadoopor KafkaKnowledge of Machine Learning concepts and model lifecycleExperience with ETL/ELT frameworks and data pipeline orchestrationStrong understanding of data warehousing and data lakes

Nice-to-Have Skills

Experience with ML frameworks (TensorFlowPyTorchScikit-learn)Familiarity with Airflow for workflow orchestrationKnowledge of Docker and KubernetesExperience in real-time streaming (KafkaKinesis)Exposure to CI/CD and DevOps practices

Tools & Platforms

Amazon Web ServicesS3GlueRedshiftLambdaEMRApache SparkHadoopKafkaAirflow

Required Skills

5+ years Data EngineeringPythonSQLAWS (S3/Glue/Redshift/Lambda/EMR)Apache SparkHadoopKafkaETL/ELTData WarehousingData LakesAirflow

Hard Skills

PythonSQLAmazon Web ServicesS3GlueRedshiftLambdaEMRApache SparkHadoopKafkaETL/ELTData WarehousingData Lakes

Soft Skills

CommunicationTeam collaborationProblem-solvingTime managementAttention to detail

Certifications

Preferred

AWS Certified Data Analytics – SpecialtyAWS Certified Solutions Architect – Associate

Industry & Role

Industry Consulting

Job Function Design and deliver data engineering solutions for analytics and ML workloads in a remote, contract environment.

Role Subtype Senior Data Engineer

Tech Domains Python, SQL / PostgreSQL, Amazon Web Services, Apache Spark, Hadoop, Kafka, Airflow, Docker, Kubernetes, Data Warehousing

Keywords for Your Resume

Senior Data EngineerRemotePythonSQLAmazon Web ServicesAWSS3GlueRedshiftLambdaEMRApache SparkHadoopKafkaETL/ELTData WarehousingData LakesAirflowCI/CDDevOpsKubernetesDockerReal-time streamingKinesis

Deal Breakers

Less than 5 years of data engineering experience, No AWS experience, Lack of Python/SQL proficiency

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile