About this role
This senior role involves leading the development and maintenance of complex data science applications and scalable data pipelines across distributed systems, supporting enterprise-scale data initiatives.
Key Responsibilities
- Lead data science application development
- Design scalable data pipelines
- Supervise data engineering teams
- Manage distributed data systems
- Ensure data security and compliance
Technical Overview
The technical environment includes Python, SQL, Scala, Hadoop, Spark, Kafka, NoSQL databases, and cloud-based data warehousing, with a focus on large-scale, real-time data processing.
Ideal Candidate
The ideal candidate is a senior data engineer with over 7 years of experience in software engineering, specializing in Python, SQL, and big data technologies. They should have extensive experience with distributed data systems, real-time streaming, and hold a Secret clearance, demonstrating leadership in complex data projects.
Must-Have Skills
7+ years of experience in software engineering in Python and SQL7+ years designingdevelopingand maintaining complex data science applications5+ years creating software for structured and unstructured data5+ years developing scalable ETL/ELT workflowsSecret clearanceBachelor's degree
Nice-to-Have Skills
10+ years of experience with PythonSQLScalaRHaskellRustC++CC#JuliaGoJava5+ years with AWSAzureGoogle Cloud5+ years with SparkDatabricksHadoopHiveHBaseAccumuloAWS EMRNifiLuigiDaskKafka5+ years working on real-time data and streaming applications5+ years with NoSQL databases such as ElasticsearchSolrMongoDBCassandra5+ years with data warehousing and databases like AWS RedshiftPostgreSQLMySQLOracleSnowflake5+ years with UNIX/LinuxShell scripting
Tools & Platforms
AWS EMRNifiLuigiDaskKafkaElasticsearchSolrMongoDBCassandraPostgreSQLMySQLOracleSnowflakeUNIXLinux
Required Skills
PythonSQLScalaRHaskellRustC++CC#JuliaGoJavaETLELTBig dataDistributed dataSparkDatabricksHadoopHiveHBaseAccumuloAWS EMRNifiLuigiDaskKafkaNoSQLElasticsearchSolrMongoDBCassandraData warehousingPostgreSQLMySQLOracleSnowflakeUNIXLinuxShell scripting
Hard Skills
PythonSQLScalaRHaskellRustC++CC#JuliaGoJavaData science applicationsETLELTData engineeringBig dataDistributed dataSparkDatabricksHadoopHiveHBaseAccumuloAWS EMRNifiLuigiDaskKafkaNoSQLElasticsearchSolrMongoDBCassandraData warehousingPostgreSQLMySQLOracleSnowflakeUNIXLinuxShell scripting
Soft Skills
leadershipsupervisionproject managementcollaborativeproblem-solvingteam leadership
Keywords for Your Resume
Data EngineerPythonSQLScalaRHaskellRustC++CC#JuliaGoJavaETLELTBig dataDistributed dataSparkDatabricksHadoopHiveHBaseAccumuloAWS EMRNifiLuigiDaskKafkaElasticsearchSolrMongoDBCassandraData warehousingPostgreSQLMySQLOracleSnowflakeUNIXLinuxShell scriptingSecret clearance
Deal Breakers
Less than 7 years of experience in software engineering, No Secret clearance, Lack of experience with big data tools, Not located at the specified locations
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile