Databricks Engineer -Hybrid -Montreal, QC / Mississauga, ON
About the role
Role: Databricks Engineer
Job Summary We are seeking highly skilled Azure Data Engineer with strong expertise in Databricks to join our data
team. The ideal candidate will design, implement and optimize large-scale data pipeline, ensuring
scalability, reliability and performance. This role involves working closely with multiple teams and
business stakeholders to deliver cutting-edge data solutions.
Key Responsibilities Data Pipeline Development:
- Build and maintain scalable ETL/ELT pipelines using Databricks.
- Leverage PySpark/Spark and SQL to transform and process large datasets.
- Integrate data from multiple sources including Azure Blob Storage, ADLS and other
relational/non-relational systems.
Collaboration & Analysis
- Work Closely with multiple teams to prepare data for dashboard and BI Tools.
- Collaborate with cross-functional teams to understand business requirements and
deliver tailored data solutions.
Performance & Optimization
- Optimize Databricks workloads for cost efficiency and performance.
- Monitor and troubleshoot data pipelines to ensure reliability and accuracy.
Governance & Security
- Implement and manage data security, access controls and governance standards using
Unity Catalog.
- Ensure compliance with organizational and regulatory data policies.
Deployment
- Leverage Databricks Asset Bundles for seamless deployment of Databricks jobs,
notebooks and configurations across environments.
- Manage version control for Databricks artifacts and collaborate with team to maintain
development best practices.
Technical Skills
- Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table
Triggers, Delta Live Pipelines, Databricks Runtime etc.)
- Proficiency in Azure Cloud Services.
- Solid Understanding of Spark and PySpark for big data processing.
- Experience in relational databases.
- Knowledge on Databricks Asset Bundles and GitLab.
Preferred Experience
- Familiarity with Databricks Runtimes and advanced configurations.
- Knowledge of streaming frameworks like Spark Streaming.
- Experience in developing real-time data solutions.
Certifications
- Azure Data Engineer Associate or Databricks certified Data Engineer Associate certification.
(Optional)
About W3Global
W3Global is a leading provider of end-to-end consulting services, empowering businesses to achieve their strategic goals and optimize their operations. With over 15 years of experience, we have a proven track record of delivering innovative and effective solutions across a wide range of industries.
Our Mission
At W3Global, we are committed to helping businesses of all sizes achieve their full potential. We believe that by combining our deep industry expertise with our innovative approach, we can deliver exceptional results that exceed our clients' expectations.
Our Services
We offer a comprehensive suite of consulting services designed to address the specific needs of our clients. Our services include:
Strategy Consulting: We help businesses develop and implement winning strategies to achieve their long-term goals.
Business Process Improvement: We identify and optimize business processes to enhance efficiency, reduce costs, and improve customer satisfaction.
Technology Consulting: We provide expert guidance on technology selection, implementation, and integration to support business objectives.
Talent Acquisition: We help businesses find and hire the right talent to support their growth and success.
Human Capital Management: We provide comprehensive HR solutions to help businesses manage their workforce effectively.
Databricks Engineer -Hybrid -Montreal, QC / Mississauga, ON
About the role
Role: Databricks Engineer
Job Summary We are seeking highly skilled Azure Data Engineer with strong expertise in Databricks to join our data
team. The ideal candidate will design, implement and optimize large-scale data pipeline, ensuring
scalability, reliability and performance. This role involves working closely with multiple teams and
business stakeholders to deliver cutting-edge data solutions.
Key Responsibilities Data Pipeline Development:
- Build and maintain scalable ETL/ELT pipelines using Databricks.
- Leverage PySpark/Spark and SQL to transform and process large datasets.
- Integrate data from multiple sources including Azure Blob Storage, ADLS and other
relational/non-relational systems.
Collaboration & Analysis
- Work Closely with multiple teams to prepare data for dashboard and BI Tools.
- Collaborate with cross-functional teams to understand business requirements and
deliver tailored data solutions.
Performance & Optimization
- Optimize Databricks workloads for cost efficiency and performance.
- Monitor and troubleshoot data pipelines to ensure reliability and accuracy.
Governance & Security
- Implement and manage data security, access controls and governance standards using
Unity Catalog.
- Ensure compliance with organizational and regulatory data policies.
Deployment
- Leverage Databricks Asset Bundles for seamless deployment of Databricks jobs,
notebooks and configurations across environments.
- Manage version control for Databricks artifacts and collaborate with team to maintain
development best practices.
Technical Skills
- Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table
Triggers, Delta Live Pipelines, Databricks Runtime etc.)
- Proficiency in Azure Cloud Services.
- Solid Understanding of Spark and PySpark for big data processing.
- Experience in relational databases.
- Knowledge on Databricks Asset Bundles and GitLab.
Preferred Experience
- Familiarity with Databricks Runtimes and advanced configurations.
- Knowledge of streaming frameworks like Spark Streaming.
- Experience in developing real-time data solutions.
Certifications
- Azure Data Engineer Associate or Databricks certified Data Engineer Associate certification.
(Optional)
About W3Global
W3Global is a leading provider of end-to-end consulting services, empowering businesses to achieve their strategic goals and optimize their operations. With over 15 years of experience, we have a proven track record of delivering innovative and effective solutions across a wide range of industries.
Our Mission
At W3Global, we are committed to helping businesses of all sizes achieve their full potential. We believe that by combining our deep industry expertise with our innovative approach, we can deliver exceptional results that exceed our clients' expectations.
Our Services
We offer a comprehensive suite of consulting services designed to address the specific needs of our clients. Our services include:
Strategy Consulting: We help businesses develop and implement winning strategies to achieve their long-term goals.
Business Process Improvement: We identify and optimize business processes to enhance efficiency, reduce costs, and improve customer satisfaction.
Technology Consulting: We provide expert guidance on technology selection, implementation, and integration to support business objectives.
Talent Acquisition: We help businesses find and hire the right talent to support their growth and success.
Human Capital Management: We provide comprehensive HR solutions to help businesses manage their workforce effectively.