Position Details
About this role
This role involves leading the development of Spark™ Structured Streaming, enhancing its capabilities, and ensuring high performance. It requires deep expertise in distributed systems, data processing, and team leadership.
Key Responsibilities
- Lead Spark™ Structured Streaming development
- Improve engine performance
- Manage technical architecture
- Collaborate with open source community
- Drive product adoption
Technical Overview
The role focuses on building and optimizing stream processing engines within the Spark™ ecosystem, managing technical debt, and driving adoption of high-performance data processing solutions.
Ideal Candidate
The ideal candidate is a senior engineer with 5+ years experience in distributed data systems, specializing in Apache Spark™ and stream processing. They have a strong background in building scalable, high-performance data engines and leading technical teams.
Must-Have Skills
Nice-to-Have Skills
Tools & Platforms
Required Skills
Hard Skills
Soft Skills
Industry & Role
Keywords for Your Resume
Deal Breakers
Lack of experience with Apache Spark™ or stream processing, No leadership experience in engineering teams, Unfamiliarity with distributed systems
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile