Position Details

Type Not Specified

Experience senior

Exp. Years 2+ years

Education Bachelor's degree in Computer Science, Information Technology, Computer Engineering or related field

Category Software Engineering

About this role

This role involves developing and maintaining data pipelines using StreamSets, designing metadata models, and ensuring data quality and integration across systems. The engineer collaborates with technical teams to optimize data workflows.

Key Responsibilities

Build ETL pipelines
Design metadata models
Ensure data quality
Collaborate on data integration
Document data processes

Technical Overview

The technical environment includes StreamSets for ETL, Snowflake and SQL databases, with a focus on data transformation, profiling, and metadata management within an agile framework.

Ideal Candidate

The ideal candidate is a software development engineer with at least 2 years of experience in building ETL pipelines, data transformation, and data quality management. They should be familiar with StreamSets, Snowflake, and SQL, and comfortable working in an agile environment.

Must-Have Skills

ETL processesStreamSetsSQLData transformationData qualityData integrationData profilingAgile methodologiesGitHubJIRA

Nice-to-Have Skills

SnowflakeOracleMS SQLSparkSQLHiveImpalaMetadata modelingData lineageData governance

Tools & Platforms

StreamSetsSnowflakeOracleMS SQLSparkSQLHiveImpalaGitHubJIRAConfluence

Required Skills

ETL processesStreamSetsSnowflakeSQLmetadata modelingdata transformationdata qualitydata integrationdata profilingagile methodologies

Hard Skills

ETL processesStreamSetsSnowflakeSQLMetadata modelingData transformationData qualityData integrationData profilingAgile methodologiesGitHubJIRAConfluenceOracleMS SQLSparkSQLHiveImpala

Soft Skills

CommunicationCollaborationProblem-solvingDocumentationTraining

Industry & Role

Industry Fintech

Job Function Data pipeline development and data management

Role Subtype Data Engineer

Tech Domains StreamSets, Snowflake, SQL / PostgreSQL, Oracle, MS SQL, SparkSQL, Hive, Impala

Clearance & Visa

Clearance Required None

Visa Sponsorship No

Keywords for Your Resume

Software Development EngineerETL processesStreamSetsSnowflakeSQLMetadata modelingData transformationData qualityData integrationData profilingAgileGitHubJIRAConfluenceData pipelineData lineageData governanceOracleMS SQLSparkSQLHiveImpalaetl processesstreamsetssnowflakesqlmetadata modelingdata transformationdata qualitydata integrationagilegitHub

Deal Breakers

Less than 2 years of experience, Lack of experience with StreamSets or Snowflake, No knowledge of SQL or data transformation, Unfamiliar with agile methodologies

Apply for this Position →

Get matched to jobs like this

Luna finds roles that fit your skills and career goals — no endless scrolling required.

Create a Free Profile

Software Development Engineering -Sr Professional

Get matched to jobs like this