Position Details

Type Full-Time

Experience senior

Exp. Years 5+ years

Education Not specified

Category Data & Analytics

About this role

Senior Data Engineer responsible for designing and building scalable batch and streaming data pipelines and the data platform powering Socure’s identity verification and analytics; lead end-to-end data initiatives and partner with cross-functional teams.

Key Responsibilities

Design and build batch and streaming data pipelines
Own end-to-end delivery of data initiatives
Develop and evolve the data platform for large-scale processing
Automate data operations (validation, quality checks, alerting, backfills)
Partner with cross-functional teams to translate requirements into technical solutions

Technical Overview

Stack includes Python/Scala on AWS with Spark; batch and streaming data processing; data modeling/warehousing; production pipelines; CI/CD; monitoring; GenAI-assisted ingestion; cloud-native tooling.

Ideal Candidate

The ideal candidate is a senior data engineer with 5+ years building production-grade data platforms. They excel at Python/Scala, Spark-based processing, and leading end-to-end data initiatives in a cloud-native environment, with strong collaboration across Data Science, Product, and Engineering.

Must-Have Skills

5+ years hands-on data engineeringPython or ScalaSQLApache SparkAWS (EMRLambdaS3)data modeling / data warehousingproduction pipelinesmonitoring / alerting / incident responseCI/CDversion control

Nice-to-Have Skills

KafkaKinesisAirflowStep FunctionsData LakehouseData MeshGenAI toolingGenAI prompt engineeringmentoring

Tools & Platforms

Apache SparkAirflowKafkaKinesisAmazon Web ServicesAWS EMRAWS LambdaAmazon S3Data LakehouseData Mesh

Required Skills

5+ years data engineering; Python/Scala; SQL; Spark; AWS (EMRLambdaS3); data modeling/warehousing; production pipelines; monitoring/alerting; CI/CD; version control; cross-functional collaboration

Hard Skills

PythonScalaSQLApache SparkAWSAmazon Web ServicesAWS EMRAWS LambdaAmazon S3Data PipelinesBatch ProcessingStreaming DataAirflowKafkaKinesisData LakehouseData MeshGenAIGenAI-assisted ingestionCI/CDGitMonitoringAlertingProduction-grade data platformscloud-native

Soft Skills

communicationcollaborationownershipproblem solvingcross-functional partnership

Industry & Role

Industry Fintech

Job Function Design and build scalable data platforms and pipelines powering Socure's identity verification products and analytics.

Role Subtype Senior Data Engineer

Tech Domains Python, SQL, Apache Spark, Amazon Web Services, Airflow, Kafka, Kinesis, Amazon S3, Data Warehousing, Data Modeling

Clearance & Visa

Visa Sponsorship No

Keywords for Your Resume

Senior Data EngineerPythonScalaSQLApache SparkAWSAmazon Web ServicesEMRLambdaS3Data PipelinesBatch ProcessingStreaming DataAirflowKafkaKinesisData LakehouseData MeshGenAIGenAI-assisted ingestionCI/CDGitMonitoringAlertingproduction-grade data platformscloud-nativedata warehousingData ModelingData Warehousing

Deal Breakers

Sponsorship required

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile