About this role
This role involves designing and developing scalable data pipelines and solutions using big data tools and cloud platforms to support enterprise data needs.
Key Responsibilities
- Designing data pipelines
- Implementing real-time streaming solutions
- Collaborating with cross-functional teams
- Optimizing data storage and retrieval
- Mentoring junior engineers
Technical Overview
Technical scope includes big data processing with Apache Flink, Java, Scala, Python, and cloud data warehousing with Snowflake, Redshift, on AWS, Azure, and Google Cloud environments.
Ideal Candidate
The ideal candidate is a mid-level data engineer with 2+ years of experience in big data technologies, cloud platforms like AWS, and proficiency in Java, Python, and NoSQL databases. They are passionate about data engineering, cloud solutions, and scalable data processing.
Must-Have Skills
Bachelor's Degreeat least 2 years of experience in big data technologiesat least 1 year experience with cloud computing (AWSMicrosoft AzureGoogle Cloud)
Nice-to-Have Skills
7+ years of application development including PythonSQLScalaor Java4+ years of experience with a public cloud (AWSMicrosoft AzureGoogle Cloud)4+ years experience with Distributed data/computing tools (MapReduceHadoopHiveEMRKafkaSparkGurobior MySQL)4+ years of experience with NoSQL implementation (MongoCassandra)4+ years of data warehousing experience (Redshift or Snowflake)4+ years of experience with UNIX/Linux including basic commands and shell scripting2+ years of experience with Agile engineering practices
Tools & Platforms
Apache FlinkJavaScalaPythonRedshiftSnowflakeAWSMicrosoft AzureGoogle CloudHadoopHiveKafkaSparkMongoDBCassandraUNIX/LinuxShell scripting
Required Skills
Apache FlinkJavaScalaPythonRedshiftSnowflakeAWSMicrosoft AzureGoogle CloudHadoopHiveKafkaSparkMongoDBCassandraUNIX/LinuxShell scriptingAgile
Hard Skills
Apache FlinkApache FlinkJavaScalaPythonOpen Source RDBMSNoSQL databasesRedshiftSnowflakeCloud computingAmazon Web ServicesAWSMicrosoft AzureGoogle CloudDistributed dataMapReduceHadoopHiveKafkaSparkMongoDBCassandraUNIX/LinuxShell scriptingAgile
Soft Skills
collaborationteamworkmentoringcommunicationproblem-solvinglearningtech trend awareness
Keywords for Your Resume
Data EngineerApache FlinkJavaScalaPythonSnowflakeRedshiftAWSAmazon Web ServicesCloud computingbig dataDistributed dataHadoopKafkaSparkMongoDBCassandraUNIX/LinuxShell scriptingAgile
Deal Breakers
Lack of experience with big data tools, No cloud computing experience, No proficiency in Java or Python, Bachelor's degree not required
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile