About this role
Databricks AI Developer / Architect to design and build Azure-based data platforms, ML pipelines, and Delta Lake architectures at scale.
Key Responsibilities
- Databricks Engineering & Architecture
- Azure cloud integration
- AI/ML pipelines & MLOps
- Data governance & security
- Collaboration with data scientists and stakeholders
Technical Overview
Hands-on Databricks with PySpark and Delta Lake; Azure cloud stack; AI/ML pipelines, MLOps, Model Serving, and GenAI integrations.
Ideal Candidate
The ideal candidate is a senior Databricks/Azure engineer and AI architect who can design end-to-end data platforms and ML pipelines using Databricks, PySpark, and Delta Lake on Azure, with ML deployment and MLOps experience.
Must-Have Skills
5–12+ years in Data EngineeringCloudor Analytics roles.Strong hands-on experience with DatabricksPySparkDelta LakeMLflowand Lakehouse architecture.Deep expertise in Azure (ADLSADFSynapseFunctionsKey VaultDevOps).Experience building AI/ML pipelinesmodel deploymentand MLOps workflows.Strong SQLPythonSpark optimizationand distributed computing skills.Experience with CI/CDGitTerraformor Infrastructure-as-Code.Understanding of data governancesecurityand enterprise data management.
Nice-to-Have Skills
Experience with Azure OpenAIDatabricks Model ServingVector Searchor LLM-based applications.Knowledge of streaming technologies (KafkaEvent HubStructured Streaming).Certifications in DatabricksAzureor AI/ML.Experience in large-scale enterprise or multi-cloud environments.
Tools & Platforms
DatabricksAzure Data Lake StorageAzure SynapseAzure SQLAzure FunctionsEvent HubKey VaultAzure DevOpsDatabricks Model ServingMLOps
Required Skills
DatabricksPySparkDelta LakeMLflowLakehouse architectureDelta Live TablesAuto LoaderAzure Data Lake StorageAzure SynapseAzure SQLAzure FunctionsEvent HubKey VaultAzure DevOpsMLOpsAI pipelinesModel ServingAzure OpenAIBedrockGenAISQLPythonSparkCI/CDTerraformInfrastructure-as-Codedata governancesecurityenterprise data management
Hard Skills
DatabricksPySparkDelta LakeMLflowLakehouse architectureDelta Live TablesAuto LoaderAzure Data Lake StorageADLSAzure SynapseAzure SQLAzure FunctionsEvent HubKey VaultAzure DevOpsMLOpsModel ServingAzure OpenAIBedrockGenAISQLPythonSparkCI/CDGitTerraformInfrastructure-as-Codedata governancesecurityenterprise data management
Soft Skills
leadershipcollaborationcommunicationproblem solvingstakeholder management
Certifications
Preferred
Databricks CertificationAzure CertificationAI/ML Certification
Keywords for Your Resume
DatabricksPySparkDelta LakeMLflowLakehouse architectureDelta Live TablesAuto LoaderAzure Data Lake StorageAzure SynapseAzure SQLAzure FunctionsEvent HubKey VaultAzure DevOpsMLOpsAI pipelinesModel ServingAzure OpenAIBedrockGenAISQLPythonSparkCI/CDTerraformInfrastructure as Codedata governancesecurityenterprise data managementAzure
Deal Breakers
Lack of Databricks or Azure experience, No experience with AI/ML pipelines
Get matched to jobs like this
Luna finds roles that fit your skills and career goals — no endless scrolling required.
Create a Free Profile