Position Details

Salary $120K – $160K USD / year

Type Full-Time

Experience senior

Exp. Years 8 years

Education Bachelor’s degree in Computer Science, Computer Engineering or related field

Category Data & Analytics

About this role

Senior data/platform engineer focused on building and maintaining data pipelines and cloud infrastructure for a remote, SaaS-based product with SOC2 considerations.

Key Responsibilities

Build and maintain PySpark data pipelines in the Databricks environment
Optimize Spark jobs performance and resource usage
Design, develop, and maintain backend software components and services
Research and build proofs of concepts in the data space
Create and maintain comprehensive technical documentation

Technical Overview

Stack includes PySpark/Databricks, PostgreSQL, GCP/AWS, Terraform, Docker, GitLab CI/CD, Keycloak/OIDC, IAM, SOC2 controls; emphasis on end-to-end data delivery, security, and documentation.

Ideal Candidate

The ideal candidate is a senior data/platform engineer with strong PostgreSQL and Terraform experience, comfortable building multi-environment cloud infrastructure (GCP/AWS) and managing end-to-end data pipelines in a SaaS environment.

Must-Have Skills

PostgreSQLTerraformDockerCI/CD pipelinesGitLab CI/CDGoogle Cloud Platform (GCP)Amazon Web Services (AWS)Keycloak / OIDCIAMSecrets managementSOC2 complianceSelf-directed / ownership mentality

Nice-to-Have Skills

Production system ownershipStartup environmentTerminal/CLI-first approach

Tools & Platforms

PostgreSQLGoogle Cloud PlatformAmazon Web ServicesKeycloakOIDCGitLab CI/CDTerraform

Required Skills

Strong PostgreSQL expertise; Terraform; Docker; CI/CD pipelines; GitLab CI/CD; Keycloak; OIDC; IAM; Secrets management; Google Cloud Platform; AWS; SOC2; Production system; ETL/ELT

Hard Skills

PostgreSQLTerraformDockerCI/CDGitLab CI/CDKeycloakOIDCIAMSecrets managementGoogle Cloud PlatformAmazon Web ServicesAWSCloud infrastructureSOC2Containerized services

Soft Skills

AutonomousStrong communicationOwnership mentalityCollaborationProblem-solving

Certifications

Preferred

Google Cloud Certified - Professional Cloud ArchitectAWS Certified Solutions Architect – Associate

Industry & Role

Industry SaaS

Job Function Data Engineer II focused on PySpark/Databricks in AWS/GCP for government/public sector programs

Role Subtype Data Platform Engineer

Tech Domains PySpark, Databricks, SQL / PostgreSQL, Google Cloud Platform, Amazon Web Services, S3, EC2, Glue, Lambda

Keywords for Your Resume

data engineer iipysp arkpysparkdatabricksgoogle cloud platformgcpterraformdockergitlab ci/cdkeycloakoidciamsecrets managementsoc2etletl/eltproduction systemcloud infrastructureawss3DatabricksPySparkGoogle Cloud PlatformGCPTerraformDockerGitLab CI/CDPostgreSQLKeycloakOIDC

Deal Breakers

Lack of 8+ years of experience, No PySpark/Databricks experience, No PostgreSQL or cloud infrastructure experience

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Senior Data / Platform Engineer

Get matched to jobs like this