Azure Data Engineer

Job summary

Build and orchestrate data pipelines using Azure Data Factory and Databricks, loading data from multiple sources into a medallion architecture and designing secure Delta tables. Configure Unity Catalog and data governance, migrate workspaces from Hive Metastore, and support deployment, monitoring, security, and optimization across Azure data services.

Job details

🔍 WE'RE HIRING: Sr Azure Data Engineer 📍 Location: Toronto, ON – Hybrid (in the office Thursday and 4th Friday of the month) 📅 Duration: 6+ Months with Possible extension Job Description: • 7+ Years experience with ADF, Databricks and Azure Functions (C#) • In depth development experience python/notebook on Databricks/Spark/panda • Expert working knowledge of Unity Catalog and data governance setup/configuration • Experience in loading datasets from various sources using ADF to Databricks with Medallion architecture • Experience on converting Databricks workspace from hive metastore to unity catalog & setup associated permissions • Creating delta table schema/tables with proper design/properties & security permissions • Nice to have experience on Delta Live Table. • DevOps & deployment pipeline using Azure DevOps/GitHub Enterprise for both ADF and databricks • In depth experience on Azure Entra ID for SSO/RBAC setup • In depth experience on working with/deploying to Azure PaaS resources such as Azure SQL PaaS, ADF/SHIR, ADLS, Databricks, Synapse dedicated SQL databases Must Have Requirements: •ADF orchestrations with Databricks • Experience with set up and configuration of data governance •Databricks (Python notebooks, jobs, clusters, Delta Lake, medallion architecture, performance tuning/cost optimization) •CI/CD, security •Monitoring

What you’ll do

Build and orchestrate data pipelines using Azure Data Factory and Databricks, loading data from multiple sources into a medallion architecture and designing secure Delta tables. Configure Unity Catalog and data governance, migrate workspaces from Hive Metastore, and support deployment, monitoring, security, and optimization across Azure data services.

Requirements

Requires 7+ years of experience with Azure Data Factory, Databricks, and Azure Functions, including strong Python, notebook, Spark, and Pandas development skills. Candidates must have hands-on expertise in Unity Catalog and data governance, Azure identity and access controls, CI/CD, monitoring, and Azure PaaS deployments.

Listed skills

• Python · Preferred
• CI/CD · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Azure Data Factory
• Databricks
• Azure Functions
• Python
• Apache Spark
• Pandas
• Unity Catalog
• Data Governance
• Medallion Architecture
• Delta Lake
• Azure DevOps
• GitHub Enterprise
• Microsoft Entra ID
• Role-Based Access Control
• CI/CD
• Performance Tuning

Job areas

• Data & Analytics
• Technology
• Software

More jobs from Experis Canada