About this role
Data & Analytics Engineering role at Associate level, responsible for building and maintaining big data pipelines, AI-assisted tooling, and scalable data processing for enterprise analytics.
Key Responsibilities
- Support the design and development of big data pipelines
- Develop and maintain Python-based data processing components
- Utilize AI-assisted development tools to improve productivity and code quality
- Monitor data pipelines for quality, reliability, and observability
- Collaborate with senior engineers and analysts on scalable data models and ingestion patterns
Technical Overview
Stack includes Python, Apache Spark, distributed systems, cloud platforms (AWS/Azure/GCP), Linux/UNIX, CI/CD, and data modeling with a focus on data pipelines and observability.
Ideal Candidate
The ideal candidate is an associate-level data engineer with 3+ years in data engineering/big data, fluent in Python and Apache Spark, and familiar with cloud platforms (AWS/Azure/GCP). They should be comfortable working on AI-assisted development tools and building scalable data pipelines.
Must-Have Skills
Bachelor's degree in computer scienceEngineeringor a related field (or equivalent practical experience)3+ years of experience or strong academic/project experience in data engineering or big data systemsWorking knowledge of Python for data processing and automationWorking knowledge of Apache SparkUnderstanding of distributed systems fundamentalsFamiliarity with cloud platforms (AWSAzureor GCP)Exposure to AI-powered developer toolsStrong problem-solving skills and willingness to learnComfortable working in Linux/UNIX environmentsCI/CD activities for data pipelines
Nice-to-Have Skills
Experience with KafkaRedisBig Data technologiesData quality and observabilityData modelingIngestion patternsStreaming data
Tools & Platforms
PythonApache SparkLinux/UNIXAWSAzureGoogle Cloud PlatformCI/CD
Required Skills
['Python''SQL / PostgreSQL''Apache Spark''Distributed systems''Big data''Data pipelines''Cloud platforms''AWS''Azure''Google Cloud Platform''Linux/UNIX''CI/CD''AI-assisted development tools''Code quality''Observability''Data models''Ingestion''Batch processing''Streaming''Data processing']
Hard Skills
PythonApache SparkDistributed systemsBatch processingStreaming data ingestionData pipelinesData processingData modelsCI/CDLinux/UNIXCloud platforms (AWSAzureGCP)AI-powered development toolsCode qualityObservabilitySQL / PostgreSQL
Soft Skills
Problem-solvingWillingness to learnCollaborationCommunicationAdaptability
Keywords for Your Resume
data engineerassociatepythonapache sparkdistributed systemsbig datadata pipelinescloud platformsawsazuregoogle cloud platformgcplinuxunixci/cdai-assisted development toolscode qualityobservabilitydata modelsingestionbatch processingstreamingdata processing
Deal Breakers
Bachelor's degree required, 3+ years data engineering/big data experience
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile