Position Details

Salary $98K – $115K USD / year

Type Not Specified

Experience mid

Exp. Years 3-5 years

Education Bachelor's degree, or equivalent work experience

Category Data & Analytics

About this role

This role involves designing and implementing scalable data lake solutions, developing data pipelines, and managing data security and governance in cloud environments, primarily using Snowflake, Databricks, and Azure.

Key Responsibilities

Design scalable data lake solutions
Develop and optimize data pipelines
Manage data governance and security
Troubleshoot cluster and server issues
Migrate ETL jobs to cloud platforms

Technical Overview

The technical environment includes cloud data platforms like Snowflake and Azure Synapse, big data tools such as Hadoop, Hive, Spark, and security tools like HashiCorp Vault and CyberArk. Scripting in Python and shell is required for automation.

Ideal Candidate

The ideal candidate is a mid-level data engineer with 3+ years of experience in data lake solutions, data pipeline development, and cloud migration, particularly with Azure and Snowflake. Strong scripting and security management skills are essential.

Must-Have Skills

Design and implement scalable data lake solutionsDevelop and optimize data pipelinesManage data governancequalityand securityTroubleshoot Linux servers and cluster issuesImplement security measures with KerberosApache RangerAtlasSetup and manage HashiCorp Vault and CyberArkMigrate ETL jobs to Azure cloud servicesDevelop automation scripts in shellPython

Nice-to-Have Skills

Azure Data FactoryAzure DatabricksAzure Synapse AnalyticsAzureCloud migrationPerformance tuningCI/CD pipelines

Tools & Platforms

SnowflakeDatabricksAzure Synapse AnalyticsHashiCorp VaultCyberArkLinuxHadoopHiveSparkAzure

Required Skills

Data EngineerSnowflakeDatabricksdata pipelinesdata governancedata securityLinuxHadoopHiveSparkAzure Synapse AnalyticsHashiCorp VaultCyberArkPythonShell scriptingCI/CD

Hard Skills

SnowflakeDatabricksData pipelinesData governanceData securityLinuxHadoopHiveSparkAzure Synapse AnalyticsHashiCorp VaultCyberArkShell scriptingPythonCI/CD

Soft Skills

collaborationproblem-solvingcommunicationteamworkautomation

Industry & Role

Industry Financial Services

Job Function Data engineering and cloud data platform management

Role Subtype Data Engineer

Tech Domains Azure, Linux, Hadoop, Databricks, Snowflake, CyberArk, HashiCorp Vault

Keywords for Your Resume

Data EngineerSnowflakeDatabricksData pipelinesData governanceData securityLinuxHadoopHiveSparkAzure Synapse AnalyticsHashiCorp VaultCyberArkPythonShell scriptingCI/CDdata engineersnowflakedatabricksdata pipelinesdata governancedata securitylinuxhadoopsparkpython

Deal Breakers

Lack of experience with Snowflake or Databricks, No cloud migration experience, No scripting skills in Python or shell, No experience with data security tools, Bachelor's degree not in relevant field

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile