Position Details

Type Full-Time

Experience mid

Exp. Years 6+ years

Education Not specified

Category Data & Analytics

About this role

This role involves working on big data projects, designing data architectures, and deploying AI and analytics solutions using Databricks and Apache Spark.

Key Responsibilities

Work on customer data projects
Design and build reference architectures
Implement big data and AI applications
Provide technical support and feedback
Manage project scope and timelines

Technical Overview

Expertise in data engineering, distributed computing with Apache Spark, cloud platforms (AWS, Azure, GCP), and MLOps for deploying scalable data solutions.

Ideal Candidate

The ideal candidate is a senior data engineer with over 6 years of experience in data platforms, proficient in Python or Scala, and experienced with distributed computing frameworks like Apache Spark. They should have strong knowledge of cloud ecosystems (AWS, Azure, GCP) and a background in technical project delivery and customer engagement.

Must-Have Skills

6+ years experience in data engineeringPython or ScalaCloud ecosystems (AWSAzureGCP)Distributed computing with Apache SparkCI/CDMLOpsDesign and deployment of data architecturesTechnical project delivery

Nice-to-Have Skills

Reference architecturesCustomer engagementBig DataAI applications

Tools & Platforms

Apache SparkAWSAzureGCPCI/CD toolsDatabricks platform

Required Skills

PythonScalaApache SparkSparkCI/CDMLOpsData engineeringData scienceData architecturesDistributed computingCloud ecosystemsAWSAzureGCPBig DataReference architecturesCustomer engagementTechnical project delivery

Hard Skills

Soft Skills

customer servicecommunicationconflict managementwhite-boardingproject management

Certifications

Required

Databricks Certification

Industry & Role

Industry Technology

Job Function Leading big data and AI project delivery using Databricks and Spark

Keywords for Your Resume

Data EngineeringPythonScalaApache SparkSparkCI/CDMLOpsDistributed computingCloud ecosystemsAWSAzureGCPBig DataReference architecturesCustomer engagementTechnical project deliveryDatabricks

Deal Breakers

Less than 6 years of experience in data engineering, No experience with Apache Spark or distributed computing, Lack of cloud ecosystem knowledge (AWS, Azure, GCP), No experience with CI/CD or MLOps, Inability to work on customer-facing projects

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Resident Solutions Architect

Get matched to jobs like this