Position Details

Type Not Specified

Experience mid

Exp. Years 6+ years

Education Not specified

Category Data & Analytics

About this role

This role involves working with clients on big data projects using the Databricks platform, providing data engineering, data science, and cloud solutions to optimize data value.

Key Responsibilities

Designing reference architectures
Building customer use cases
Implementing big data solutions
Supporting operational issues
Collaborating with technical teams

Technical Overview

Technical scope includes data architecture design, distributed computing with Apache Spark, cloud integrations (AWS, Azure, GCP), and MLOps, primarily on the Databricks platform.

Ideal Candidate

The ideal candidate is a mid-level data engineer with over 6 years of experience in data platforms and analytics, proficient in Python or Scala, with deep expertise in Apache Spark and cloud ecosystems like AWS, Azure, or GCP. They should have strong project delivery skills and experience working directly with clients on big data solutions.

Must-Have Skills

6+ years experience in data engineeringdata platforms & analyticsComfortable writing code in either Python or ScalaWorking knowledge of two or more cloud ecosystems (AWSAzureGCP)Deep experience with distributed computing with Apache SparkKnowledge of Spark runtime internalsFamiliarity with CI/CD for production deploymentsWorking knowledge of MLOpsDesign and deployment of performant end-to-end data architecturesExperience with technical project deliveryDocumentation and white-boarding skillsExperience working with clients and managing conflicts

Nice-to-Have Skills

Databricks Certification

Tools & Platforms

Databricks platformApache SparkAWSAzureGCPCI/CD toolsMLOps tools

Required Skills

Data engineeringPythonScalaCloud ecosystemsAWSAzureGCPApache SparkSpark runtime internalsCI/CDMLOpsData platformsData scienceDistributed computing

Hard Skills

PythonScalaCloud ecosystemsAWSAzureGCPApache SparkSpark runtime internalsCI/CDMLOpsData engineeringData platformsData scienceDistributed computing

Soft Skills

Customer serviceCommunicationProject deliveryWhite-boardingConflict managementClient engagement

Certifications

Preferred

Databricks Certification

Industry & Role

Industry SaaS

Job Function Big Data Solutions Architect

Keywords for Your Resume

Data EngineerData PlatformsData ScienceApache SparkSpark runtime internalsPythonScalaAWSAzureGCPDistributed computingMLOpsCI/CDDatabricksBig DataData engineering

Deal Breakers

Less than 6 years experience in data engineering, No experience with Apache Spark or Databricks, Lack of cloud ecosystem knowledge, Inability to work with clients or manage conflicts

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Resident Solutions Architect

Get matched to jobs like this