Position Details

Salary $77K – $176K USD / year

Type Not Specified

Experience senior

Exp. Years 5+ years

Education Bachelor's degree

Category Data & Analytics

About this role

This role involves developing and managing large-scale Spark-based data pipelines for classified government projects, ensuring secure and efficient data processing.

Key Responsibilities

Develop Spark data pipelines
Manage petabyte-scale data processing
Lead data engineering teams
Implement data security protocols
Collaborate on data architecture

Technical Overview

The technical environment includes Spark, PySpark, Java Spark, AWS cloud services, Kafka, NoSQL databases like Cassandra, PostgreSQL, and containerization with Docker and Kubernetes.

Ideal Candidate

The ideal candidate is a senior Spark data engineer with over 5 years of experience in big data processing, ETL pipeline development, and distributed systems. They should hold TS/SCI clearance and have expertise in NoSQL databases and containerization technologies.

Must-Have Skills

5+ years of experience in Spark developmentExperience with big data and ETL pipelinesExperience with distributed data toolsExperience with NoSQL databasesExperience with data warehousingExperience with containerization technologyExperience with Agile practicesTS/SCI clearance with a polygraphBachelor's degree

Nice-to-Have Skills

Experience with large-scale Postgres databasesExperience with graph databases and data modelingExperience with containerization (DockerKubernetes)

Tools & Platforms

SparkPySparkJava SparkAWSKafkaCassandraPostgreSQLGraph databasesDockerKubernetes

Required Skills

SparkPySparkJava SparkBig dataETL pipelinesPetabyte scale dataData modelingDistributed dataAWSKafkaNoSQLCassandraPostgreSQLGraph databasesDockerKubernetes

Hard Skills

SparkPySparkJava SparkBig dataETL pipelinesPetabyte scale dataData modelingDistributed dataAWSKafkaNoSQL databasesCassandraPostgreSQLGraph databasesDockerKubernetesAgile engineering

Soft Skills

team leadershipmentoringcollaborationproblem-solvingcommunication

Certifications

Required

TS/SCI clearance with a polygraph

Industry & Role

Industry Government/Public Sector

Job Function Senior Spark data engineering for classified government data projects

Clearance & Visa

Clearance Required TS/SCI clearance with a polygraph

Keywords for Your Resume

SparkPySparkJava SparkBig dataETL pipelinesPetabyte scale dataData modelingDistributed dataAWSKafkaNoSQLCassandraPostgreSQLGraph databasesDockerKubernetesAgile engineeringTS/SCI clearancePolygraph

Deal Breakers

Lack of Spark or big data experience, No TS/SCI clearance with polygraph, No Bachelor's degree, Inability to work in a hybrid environment

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Spark Data Engineer, Senior

Get matched to jobs like this