About the role
Job Details:
Role: Databricks Engineer
Location: Mississauga, ON/Hybrid
Employment Type: Contract
Job Description:
We are seeking highly skilled Azure Data Engineer with strong expertise in Databricks to join our data team. The ideal candidate will design, implement and optimize large-scale data pipeline, ensuring scalability, reliability and performance. This role involves working closely with multiple teams and business stakeholders to deliver cutting-edge data solutions.
Key Responsibilities:
Data Pipeline Development:
- Build and maintain scalable ETL/ELT pipelines using Databricks.
- 10+ years of experience is required.
- Leverage PySpark/Spark and SQL to transform and process large datasets.
- Integrate data from multiple sources including Azure Blob Storage, ADLS and other relational/non-relational systems.
Collaboration & Analysis:
- Work Closely with multiple teams to prepare data for dashboard and BI Tools.
- Collaborate with cross-functional teams to understand business requirements and deliver tailored data solutions.
Performance & Optimization:
- Optimize Databricks workloads for cost efficiency and performance.
- Monitor and troubleshoot data pipelines to ensure reliability and accuracy.
Governance & Security:
- Implement and manage data security, access controls and governance standards using Unity Catalog.
- Ensure compliance with organizational and regulatory data policies.
Deployment:
- Leverage Databricks Asset Bundles for seamless deployment of Databricks jobs, notebooks and configurations across environments.
- Manage version control for Databricks artifacts and collaborate with team to maintain development best practices.
Technical Skills:
- Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Delta Live Pipelines, Databricks Runtime etc.)
- Proficiency in Azure Cloud Services.
- Solid Understanding of Spark and PySpark for big data processing.
- Strong programming skills in Python.
- Experience in relational databases.
- Knowledge on Databricks Asset Bundles and GitLab.
Preferred Experience:
- Familiarity with Databricks Runtimes and advanced configurations.
- Knowledge of streaming frameworks like Spark Streaming.
- Experience in developing real-time data solutions.
Certifications:
- Azure Data Engineer Associate or Databricks certified Data Engineer Associate certification. (Optional)
About LogicsT Technologies
Founded in 2017, LogicsT Technologies Pvt. Ltd.. has established itself as a leader in delivering unparalleled services across IT and non-IT sectors. With a global footprint serving over 500 clients, we pride ourselves on being the largest provider of comprehensive solutions tailored to meet diverse business needs. Our expeFounded in 2017, LogicsT Technologies has established itself as a leader in delivering unparalleled services across IT and non-IT sectors. With a global footprint serving over 500 cliertise spans various domains, including Business Process Outsourcing (BPO), Knowledge Process Outsourcing (KPO), and specialized staffing solutions for both IT and non-IT sectors. At LogicsT Technologies Pvt. Ltd.., we offer a suite of services designed to streamline your operations and enhance productivity. Our offerings include payroll solutions, compliance solutions, and Employer of Record (EOR) and Agent of Record (AOR) services, ensuring you have the support you need for seamless operations. As a Professional Employer Organization (PEO), we help businesses navigate complex employment regulations while providing top-notch staffing solutions. Our commitment to excellence extends to global payroll solutions and Statement of Work (SOW) services, enabling organizations to effectively manage their contingent workforce and strategic project staffing needs. We specialize in automated testing platforms, embedded software staffing, and technology staffing, ensuring your projects are delivered with the highest quality standards. Our customer service is unparalleled, with dedicated inbound and outbound services and efficient ticketing solutions to address all client needs. With LogicsT Technologies Pvt. Ltd.., you are not just choosing a service provider; you are partnering with a dedicated team committed to your success. Let us help you transform your business challenges into growth opportunities. Together, we can build a future that thrives on innovation and excellence.
About the role
Job Details:
Role: Databricks Engineer
Location: Mississauga, ON/Hybrid
Employment Type: Contract
Job Description:
We are seeking highly skilled Azure Data Engineer with strong expertise in Databricks to join our data team. The ideal candidate will design, implement and optimize large-scale data pipeline, ensuring scalability, reliability and performance. This role involves working closely with multiple teams and business stakeholders to deliver cutting-edge data solutions.
Key Responsibilities:
Data Pipeline Development:
- Build and maintain scalable ETL/ELT pipelines using Databricks.
- 10+ years of experience is required.
- Leverage PySpark/Spark and SQL to transform and process large datasets.
- Integrate data from multiple sources including Azure Blob Storage, ADLS and other relational/non-relational systems.
Collaboration & Analysis:
- Work Closely with multiple teams to prepare data for dashboard and BI Tools.
- Collaborate with cross-functional teams to understand business requirements and deliver tailored data solutions.
Performance & Optimization:
- Optimize Databricks workloads for cost efficiency and performance.
- Monitor and troubleshoot data pipelines to ensure reliability and accuracy.
Governance & Security:
- Implement and manage data security, access controls and governance standards using Unity Catalog.
- Ensure compliance with organizational and regulatory data policies.
Deployment:
- Leverage Databricks Asset Bundles for seamless deployment of Databricks jobs, notebooks and configurations across environments.
- Manage version control for Databricks artifacts and collaborate with team to maintain development best practices.
Technical Skills:
- Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Delta Live Pipelines, Databricks Runtime etc.)
- Proficiency in Azure Cloud Services.
- Solid Understanding of Spark and PySpark for big data processing.
- Strong programming skills in Python.
- Experience in relational databases.
- Knowledge on Databricks Asset Bundles and GitLab.
Preferred Experience:
- Familiarity with Databricks Runtimes and advanced configurations.
- Knowledge of streaming frameworks like Spark Streaming.
- Experience in developing real-time data solutions.
Certifications:
- Azure Data Engineer Associate or Databricks certified Data Engineer Associate certification. (Optional)
About LogicsT Technologies
Founded in 2017, LogicsT Technologies Pvt. Ltd.. has established itself as a leader in delivering unparalleled services across IT and non-IT sectors. With a global footprint serving over 500 clients, we pride ourselves on being the largest provider of comprehensive solutions tailored to meet diverse business needs. Our expeFounded in 2017, LogicsT Technologies has established itself as a leader in delivering unparalleled services across IT and non-IT sectors. With a global footprint serving over 500 cliertise spans various domains, including Business Process Outsourcing (BPO), Knowledge Process Outsourcing (KPO), and specialized staffing solutions for both IT and non-IT sectors. At LogicsT Technologies Pvt. Ltd.., we offer a suite of services designed to streamline your operations and enhance productivity. Our offerings include payroll solutions, compliance solutions, and Employer of Record (EOR) and Agent of Record (AOR) services, ensuring you have the support you need for seamless operations. As a Professional Employer Organization (PEO), we help businesses navigate complex employment regulations while providing top-notch staffing solutions. Our commitment to excellence extends to global payroll solutions and Statement of Work (SOW) services, enabling organizations to effectively manage their contingent workforce and strategic project staffing needs. We specialize in automated testing platforms, embedded software staffing, and technology staffing, ensuring your projects are delivered with the highest quality standards. Our customer service is unparalleled, with dedicated inbound and outbound services and efficient ticketing solutions to address all client needs. With LogicsT Technologies Pvt. Ltd.., you are not just choosing a service provider; you are partnering with a dedicated team committed to your success. Let us help you transform your business challenges into growth opportunities. Together, we can build a future that thrives on innovation and excellence.