Position Details

Salary $77K – $176K USD / year

Type Full-Time

Experience mid

Exp. Years 3+ years

Education Bachelor's degree

Category Data & Analytics

About this role

This role involves designing, building, and operating scalable data and analytics solutions using Databricks platform, supporting government missions with AI-enabled analytics and machine learning.

Key Responsibilities

Design scalable data solutions
Build and operate Databricks workflows
Integrate Databricks with CI/CD pipelines
Apply machine learning techniques
Support government data projects

Technical Overview

The technical environment includes Databricks, cloud platforms (AWS, Azure, GCP), PySpark, Spark SQL, Delta Lake, Unity Catalog, and MLflow, focusing on data engineering, ML workflows, and data architecture.

Ideal Candidate

The ideal candidate is a mid-level data engineer with 3+ years of experience in data engineering or analytics, proficient in Databricks platform, PySpark, and cloud platforms like AWS, Azure, or GCP. They should have experience with data architecture, MLflow, and working within government or regulated environments.

Must-Have Skills

3+ years of experience in data engineering or analytics engineering2+ years of experience with the Databricks platformExperience in PySpark and Spark SQLExperience with Databricks clustersjobs or workflowsDelta Lakeand Unity CatalogExperience integrating Databricks with CI / CD pipelinesKnowledge of Python for data engineering and ML workflowsKnowledge of cloud platforms such as AWSAzureor GCPAbility to operate effectively in structuredcompliance-driven government environmentsBachelor's degree

Nice-to-Have Skills

federal government experienceMLflow-based lifecycle managementstreaming frameworksdata quality frameworksobservability toolingDatabricks CertificationCloud Platform Certification

Tools & Platforms

DatabricksAWSAzureGCPMLflowDelta LakeUnity Catalog

Required Skills

DatabricksPySparkSpark SQLDelta LakeUnity CatalogCI / CD pipelinesPythonAWSAzureGCPMLflowData EngineeringMachine LearningData ArchitectureGovernment

Hard Skills

DatabricksPySparkSpark SQLDelta LakeUnity CatalogCI / CD pipelinesPythonAWSAzureGCPMLflowAIMachine LearningData EngineeringModern Data Architecture

Soft Skills

problem-solvingcommunicationteamworkdisciplinecollaboration

Certifications

Preferred

Databricks CertificationCloud Platform Certification

Industry & Role

Industry Government/Public Sector

Job Function Data engineering and analytics platform development for government applications

Role Subtype Data Engineer

Tech Domains Databricks, AWS, Azure, GCP, MLflow

Keywords for Your Resume

DatabricksPySparkSpark SQLDelta LakeUnity CatalogCI / CD pipelinesPythonAWSAzureGCPMLflowData EngineeringMachine LearningData ArchitectureGovernment

Deal Breakers

Lack of experience with Databricks platform, No experience in cloud platforms (AWS, Azure, GCP), No Bachelor's degree, Less than 2 years of experience in data engineering

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Databricks Engineer

Get matched to jobs like this