About this role
This role involves developing and maintaining scalable data pipelines and platforms for government clients, utilizing big data technologies and programming skills.
Key Responsibilities
- Developing data pipelines
- Managing large-scale data ecosystems
- Implementing ETL/ELT workflows
- Ensuring data security and compliance
- Collaborating with cross-functional teams
Technical Overview
The technical environment includes big data frameworks like Spark, Hadoop, Hive, Kafka, and cloud-based data warehousing solutions such as Redshift and Snowflake, with programming in Python, Java, and C++.
Ideal Candidate
The ideal candidate is a mid-level data engineer with 2+ years of experience in programming languages like Python, Java, or C++, and hands-on experience with big data tools such as Spark, Hadoop, and AWS EMR. They should be skilled in developing scalable data pipelines and working within government or public sector projects.
Must-Have Skills
2+ years of experience utilizing programming languagesincluding C++Javaor PythonExperience creating software for retrievingparsing and processing structured and unstructured dataExperience developing scalable ETL or ELT workflows for reporting and analyticsAbility to develop scripts and programs for converting various types of data into usable formatsAbility to obtain and maintain a Public Trust or Suitability/Fitness determination
Nice-to-Have Skills
Experience with FAA and NASApplication development utilizing SQL or ScalaExperience with distributed data or computing tools such as SparkDatabricksHadoopHiveAWS EMRor KafkaExperience working on real-time data and streaming applicationsExperience with NoSQL implementation using MongoDB or CassandraExperience with data warehousingincluding AWS RedshiftMySQLor SnowflakeExperience with UNIX or Linuxincluding basic commands and Shell scriptingExperience with Agile engineering practices
Tools & Platforms
AWSAmazon Web ServicesHadoopHiveSparkDatabricksKafkaMongoDBCassandraAWS EMRRedshiftMySQLSnowflakeUNIXLinux
Required Skills
Programming languages (C++JavaPython)ETLELTbig data tools (SparkHadoopHiveKafka)NoSQL (MongoDBCassandra)data warehousing (RedshiftMySQLSnowflake)UNIXLinuxShell scriptingAgile
Hard Skills
C++C++JavaJavaPythonPythonSQLScalaSparkDatabricksHadoopHiveAWS EMRKafkaNoSQLMongoDBCassandraData warehousingAWS RedshiftMySQLSnowflakeUNIXLinuxShell scriptingAgile
Soft Skills
collaborativeteam environmentanalytical explorationdata examination
Keywords for Your Resume
Data EngineerETLELTbig dataApache SparkHadoopHiveKafkaMongoDBCassandraAWS EMRRedshiftMySQLSnowflakePythonJavaC++ScalaUNIXLinuxShell scriptingAgilePublic TrustNAS
Deal Breakers
Lack of experience with big data tools like Spark or Hadoop, No experience with AWS EMR or data warehousing, No Bachelor's degree, Inability to obtain Public Trust clearance
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile