Position Details

Salary $86K – $113K USD / year

Type Full-Time

Experience senior

Exp. Years 3-5 years

Education Not specified

Category Data & Analytics

About this role

A senior data engineering role focused on designing and implementing scalable data processing solutions using Apache Spark, Python, and Scala to support analytics and data science teams.

Key Responsibilities

Design data pipelines
Develop scalable data processing solutions
Collaborate with data scientists
Ensure data quality
Optimize data workflows

Technical Overview

Involves big data processing, data pipeline development, and data quality assurance within a scalable environment leveraging Apache Spark, Python, and Scala.

Ideal Candidate

The ideal candidate is a senior data engineer with 3-5 years experience in big data processing, proficient in Apache Spark, Python, and Scala. They should have strong collaboration skills and experience developing scalable data pipelines.

Must-Have Skills

Apache SparkPythonScalaData pipeline development

Nice-to-Have Skills

Big DataData qualityData transformations

Tools & Platforms

Apache SparkPythonScala

Required Skills

Apache SparkPythonScalaData pipeline developmentData processingData qualityData transformationsAnalyticsBig Data

Hard Skills

Apache SparkPythonScalaData pipeline developmentData processingData qualityData transformationsAnalyticsBig Data

Soft Skills

CollaborationProblem-solvingCommunicationMentoringTeamwork

Certifications

Required

DASCA Big Data AssociateDASCA Big Data AnalystDASCA Big Data EngineerDASCA Big Data Senior Big Data AnalystDASCA Big Data Senior Big Data EngineerIIBA Business Data Analytics Certification

Industry & Role

Industry Technology/IT

Job Function Senior Data Engineer

Keywords for Your Resume

Apache SparkPythonScalaData pipelineData processingData qualityData transformationsAnalyticsBig DataData Engineer

Deal Breakers

Lack of experience with Apache Spark, No proficiency in Python or Scala, Inability to develop data pipelines

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile