Position Details

Salary $62K – $141K USD / year

Type Not Specified

Experience mid

Exp. Years 2+ years

Education Bachelor's degree in Computer Science, Engineering, Information Systems, or Mathematics

Category Data & Analytics

About this role

This role involves supporting mission-critical data initiatives by designing, building, and maintaining scalable data pipelines on the Databricks platform within a government or defense context.

Key Responsibilities

Build and optimize data pipelines
Implement data ingestion and transformation processes
Collaborate with stakeholders to understand data requirements
Support data platform operations and troubleshoot issues
Document data models and pipelines

Technical Overview

The technical environment includes Databricks, Delta Lake, PySpark, Spark SQL, cloud storage solutions like Azure Data Lake and AWS S3, and data governance tools like Unity Catalog.

Ideal Candidate

The ideal candidate is a mid-level data engineer with at least 2 years of experience in data engineering, proficient in Databricks, PySpark, and cloud environments like AWS or Azure. They should have strong skills in building scalable data pipelines and data governance.

Must-Have Skills

2+ years of experience in data engineeringExperience writing SQL queriesExperience with Python programmingKnowledge of distributed data processing conceptsAbility to work in a team environmentAbility to obtain and maintain a Public Trust or Suitability/Fitness determinationBachelor's degree in Computer ScienceEngineeringInformation Systemsor Mathematics

Nice-to-Have Skills

Experience with Databricks notebooksExperience with cloud platforms such as AzureAWSor GCPExperience with orchestration tools such as Airflow or dbtExperience with streaming data processingKnowledge of GitKnowledge of data governance concepts

Tools & Platforms

DatabricksAzureAWSGCPS3Azure Data Lake StorageUnity CatalogPhoton engine

Required Skills

DatabricksPySparkDelta LakeETLELTData pipelinesData modelingData governanceSpark SQLPhoton engineUnity CatalogCloud platformsPythonSQL

Hard Skills

DatabricksDatabricks NotebooksDelta LakePySparkSpark SQLDelta Live TablesDLTUnity CatalogAzure Data Lake StorageAWS S3PythonSQLSpark fundamentalsPhoton engineELTETLData pipelinesData modelingData governance

Soft Skills

communicationteamworkcollaborationproblem-solvingtroubleshootingknowledge transfer

Industry & Role

Industry Government/Public Sector

Job Function Data engineering and platform support for mission-critical data initiatives

Clearance & Visa

Clearance Required Public Trust or Suitability/Fitness

Keywords for Your Resume

DatabricksPySparkDelta LakeETLELTData pipelinesData modelingData governanceSpark SQLPhoton engineUnity CatalogAzure Data Lake StorageAWS S3SQLPythonData engineeringCloud platformsData platform operationsTroubleshooting

Deal Breakers

Lack of experience with Databricks or PySpark, No experience in data engineering, Inability to obtain security clearance, No Bachelor's degree

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Databricks Engineer

Get matched to jobs like this