Position Details

Type Not Specified

Experience senior

Exp. Years 12+ years

Education BS, MS, or PhD in Computer Science, Computer Engineering, or closely related field

Category Data & Analytics

About this role

This role involves leading big data processing initiatives, designing reference architectures for Spark applications, and working closely with clients to optimize data workflows and accelerate analytics using open source tools.

Key Responsibilities

Lead Spark application architecture
Define reference architectures
Engage with customers for technical solutions
Optimize data processing workflows
Collaborate with engineering teams

Technical Overview

The technical environment includes Apache Spark, Hadoop, Hive, Presto, Kafka, and RAPIDS Accelerator, with programming in Python, Scala, Java, and SQL. The focus is on scalable data processing and analytics.

Ideal Candidate

The ideal candidate is a senior data engineer or architect with over 12 years of experience in big data processing, specializing in Apache Spark and open source data ecosystems. They have a strong background in designing reference architectures and leading technical engagements with enterprise clients.

Must-Have Skills

12+ years of work or research experience in software developmentProgramming skills in PythonScalaJavaor SQLExperience with Apache Spark and open source big data frameworksAbility to define reference architecture for Spark applicationsExperience leading technical engagement with customers

Nice-to-Have Skills

Experience with RAPIDS Accelerator for Apache SparkKnowledge of open source big data ecosystemExperience with data analytics and machine/deep learningFamiliarity with cloud services

Tools & Platforms

Apache SparkHadoopHivePrestoAirflowKafkaRAPIDS AcceleratorOpen Source Big Data Tools

Required Skills

Apache SparkBig DataData ProcessingPythonScalaJavaSQLHadoopHivePrestoAirflowKafkaRAPIDS Accelerator

Hard Skills

Apache SparkBig DataData ProcessingPythonScalaJavaSQLOpen Source EcosystemHadoopHivePrestoAirflowKafkaData AnalyticsData IntegrationML/DL

Soft Skills

Problem SolvingCustomer-facing CommunicationLeadershipTechnical EngagementSolution Design

Industry & Role

Industry Technology

Job Function Design and lead big data processing architectures and client engagements

Keywords for Your Resume

Senior Application EngineerApache SparkBig DataData ProcessingSpark accelerationData analyticsML/DLOpen Source Big DataHadoopHivePrestoAirflowKafkaRAPIDS AcceleratorPythonScalaJavaSQL

Deal Breakers

Less than 12 years of experience in software development or research, Lack of experience with Apache Spark or big data frameworks, Inability to define reference architectures, No experience with customer engagement

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Senior Application Engineer - Big Data

Get matched to jobs like this