Position Details

Salary $179K – $204K USD / year

Type Full-Time

Experience lead

Exp. Years 4+ years

Education Bachelor's Degree

Category Data & Analytics

About this role

This role involves leading the development of scalable data pipelines and analytics solutions using big data and cloud technologies. The engineer will focus on real-time data processing and data warehousing to support business insights.

Key Responsibilities

Leading data pipeline development
Implementing real-time streaming
Collaborating with teams
Optimizing data workflows
Mentoring junior staff

Technical Overview

The technical environment includes cloud platforms (AWS, Azure, Google Cloud), big data tools (Hadoop, Spark, Kafka), data warehousing (Redshift, Snowflake), and programming in Python, Scala, and Java.

Ideal Candidate

The ideal candidate is a senior data engineer with over 4 years of experience in big data and cloud platforms like AWS, Azure, or Google Cloud. They should be proficient in Python, Scala, and Java, with strong knowledge of data warehousing and streaming technologies.

Must-Have Skills

Bachelor's Degreeat least 4 years of application development experienceat least 2 years of big data technologies experienceat least 1 year experience with cloud computing (AWSMicrosoft AzureGoogle Cloud)

Nice-to-Have Skills

7+ years of application development including PythonSQLScalaJava4+ years with a public cloud (AWSMicrosoft AzureGoogle Cloud)4+ years experience with MapReduceHadoopHiveEMRKafkaSparkGurobiMySQL4+ years working on real-time data and streaming applications4+ years with NoSQL (MongoCassandra)4+ years of data warehousing (RedshiftSnowflake)UNIX/Linux basic commands and shell scripting2+ years of Agile practices

Tools & Platforms

RedshiftSnowflakeHadoopHiveKafkaSparkAWSMicrosoft AzureGoogle Cloud

Required Skills

JavaScalaPythonRedshiftSnowflakeHadoopKafkaSparkMachine LearningDistributed MicroservicesAWSMicrosoft AzureGoogle Cloud

Hard Skills

JavaScalaPythonOpen Source RDBMSNoSQL databasesRedshiftSnowflakeMachine LearningDistributed MicroservicesCloud ComputingAWSMicrosoft AzureGoogle Cloud

Soft Skills

collaborativeproblem-solvingmentoringteamworkcommunicationinnovativetech trend awareness

Industry & Role

Industry Fintech

Job Function Leading big data and cloud-based data engineering projects

Role Subtype Data Engineer

Tech Domains Amazon Web Services, Microsoft 365, Azure, Python, Java, Scala, SQL / PostgreSQL, Hadoop, Kafka, Spark

Clearance & Visa

Clearance Required None

Visa Sponsorship No

Keywords for Your Resume

Data Engineerbig datacloud computingAWSMicrosoft AzureGoogle CloudPythonJavaScalaNoSQLRedshiftSnowflakeHadoopKafkaSparkMachine LearningDistributed Microservicesdata engineer

Deal Breakers

Lack of cloud experience, Less than 4 years of application development, No experience with big data tools, No proficiency in Python, Scala, or Java

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Lead Data Engineer

Get matched to jobs like this