Databricks Architect

Job summary

Design and implement scalable, secure Databricks lakehouse architectures and production data pipelines using SQL, PySpark, Python, Medallion Architecture, and Auto Loader. Optimize workloads, implement governance and security practices, integrate cloud services, and provide technical leadership and production support.

Job details

Job Title: Databricks Architect Location: Canada Work Mode: Remote Job Type: Contract / Full-Time Certification: Databricks Certification preferred Job Summary We are seeking an experienced Databricks Architect to design and implement scalable, secure, and high-performance data lakehouse solutions. The ideal candidate should have strong expertise in Databricks, Medallion Architecture, Azure/AWS cloud platforms, SQL, PySpark, Python, and Databricks Auto Loader. Key Responsibilities Design and implement enterprise-scale data architectures using Databricks. Develop scalable data pipelines using SQL, PySpark, and Python. Implement Medallion Architecture, including Bronze, Silver, and Gold layers. Build incremental and streaming data ingestion pipelines using Databricks Auto Loader. Integrate Databricks with Azure or AWS cloud services. Optimize Spark workloads, SQL queries, cluster configurations, and data processing performance. Implement Delta Lake solutions for reliable data storage and processing. Establish data governance, security, and access control best practices. Provide technical leadership and architectural guidance to data engineering teams. Support production deployments, troubleshooting, and performance optimization. Required Skills and Qualifications Strong experience with Databricks architecture and Lakehouse solutions. Hands-on expertise in Medallion Architecture. Proficiency in SQL, PySpark, and Python. Practical experience with Databricks Auto Loader. Experience with Azure Databricks or AWS Databricks. Strong understanding of Apache Spark, Delta Lake, and ETL/ELT pipelines. Experience in data pipeline optimization and production-grade data engineering. Knowledge of data governance, security, and scalable architecture design. Strong communication, analytical, and problem-solving skills. Preferred Qualifications Databricks certification preferred. Experience with Unity Catalog and data governance. Knowledge of Delta Live Tables (DLT) or Lakeflow Declarative Pipelines. Experience with CI/CD, Git, and infrastructure-as-code tools. Experience with cloud migration and enterprise data platform modernization. Previous experience leading technical teams or defining data architecture standards. Ideal Candidate The ideal candidate will be a hands-on technical architect who can design end-to-end Databricks solutions, guide engineering teams, and deliver scalable data platforms using modern cloud and lakehouse technologies.

What you’ll do

Design and implement scalable, secure Databricks lakehouse architectures and production data pipelines using SQL, PySpark, Python, Medallion Architecture, and Auto Loader. Optimize workloads, implement governance and security practices, integrate cloud services, and provide technical leadership and production support.

Requirements

Candidates should have strong hands-on experience with Databricks architecture, Medallion Architecture, SQL, PySpark, Python, Auto Loader, Apache Spark, Delta Lake, and Azure or AWS. The role also requires experience optimizing production data pipelines and designing secure, scalable platforms; Databricks certification is preferred.

Listed skills

• SQL · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Databricks Architecture
• Lakehouse Solutions
• Medallion Architecture
• SQL
• PySpark
• Python
• Databricks Auto Loader
• Azure Databricks
• AWS Databricks
• Apache Spark
• Delta Lake
• ETL/ELT Pipelines
• Data Pipeline Optimization
• Data Governance
• Unity Catalog
• Delta Live Tables

Job areas

• Technology
• Data & Analytics
• Software
• Management & Leadership
• Consulting

More jobs from Celebal Technologies