Data Platform Engineer, Fauna

Build the foundational data platform that powers robotics and machine learning development. You will design scalable ingestion, storage, processing, transformation, and real-time monitoring systems for robot telemetry, video, logs, and performance metrics.

• Design and build scalable data pipelines for ingesting and processing robotics data (sensor streams, video, telemetry, logs)
• Develop and maintain data storage solutions optimized for diverse data types and access patterns
• Create tools and APIs for researchers and engineers to efficiently query and analyze large datasets
• Build real-time data processing systems for monitoring robot fleet performance
• Build and maintain data transformation pipelines that prepare robotics data for ML training

Own end-to-end robotics data pipeline architecture: ingest (sensor telemetry/video/logs), store across diverse access patterns, transform data for ML training, and implement real-time processing for fleet performance monitoring. Work with Kafka and/or Hadoop ecosystem components (Hive, Spark, HBase, YARN) and cloud infrastructure to deliver APIs and tooling for querying and analysis at scale.

The ideal candidate is a data engineer with 3+ years of experience building scalable data pipelines for ingesting and transforming high-volume robotics data. They have strong advanced SQL and Python automation skills, plus experience with Kafka and distributed systems on cloud infrastructure. They can partner with ML and robotics teams to deliver real-time processing and well-structured datasets for machine learning training.