Principal Databricks Data Engineer
About the role
Job Title: Principal Databricks Data Engineer Location: Toronto, ON onsite Job Type: Fulltime Permanent Role
Job Summar
y **Key Requirements **12–18 years of overall data engineering experien ce8+ years of experience in enterprise Data Warehouse and Data Lake platfor ms5+ years of hands-on experience with Databricks and Spark at sca leStrong experience in modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehou seRedesign ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta La keRefactor legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelin esEnsure data reconciliation, audit integrity, and consistency during migrati onDesign and govern enterprise Data Warehouse and Data Lake/Lakehouse architectur esImplement layered architecture including Raw/Landing, Curated/Conformed, and Semantic/Consumption laye rsModernize traditional EDW platforms into scalable lakehouse architectur esStrong experience in finance and risk data models including General Ledger, Sub-ledger, financial hierarchies, and risk exposure models (credit, liquidity, market ris
k)
- Enable reporting use cases including aggregation, drill-down, and drill-back capabilit
ies
- Build and manage semantic/consumption layers for BI, reporting, and analy
tics
- Define business metrics, dimensions, hierarchies, and
KPIs
- Experience with Databricks SQL, Delta tables, and dbt or similar fram
eworks
- Develop and optimize large-scale data pipelines using PySpark, Spark SQL, and Del
ta Lake
- Implement Medallion architecture (Bronze, Silver, Gold
layers)
- Optimize workloads using Z-ORDER, OPTIMIZE, caching, and cluster confi
gurations
- Implement data governance, data quality frameworks, reconciliation controls, and exceptio
n handling
- Establish data lineage and metadata
management
- Ensure data security, access control, and complian
ce standards
- Experience with cloud platforms such as
AWS or Azure
- Experience with CI/CD pipelines using Git, Terraform, Jenkins, o
r Azure DevOps
- Familiarity with orchestration tools such as Airflow or Datab
ricks Workflows
- Experience wi
th dbt is a plus
- Act as a technical authority and lead archi
tecture decisions
- Mentor and guide senior engineers and establish eng
ineering standards
- Strong stakeholder management with finance, risk, analytics, a
nd governance teams
- Ability to translate complex data structures into bus
iness-ready insi
ghts Nice to Have
- Experience in BFSI, Capital Markets, o r regulatory reporting* Exposure to SAP Finance, Oracle Financials, or S/4HANA* Experience supp orting AI/ML workloads* Databricks o
r cloud certi fications Warm Regard s,Akanksha Singh BaghelSen ior Technical Recruite rPhone: +1-484-756-3700Email: akansha.sin gh@zodiac-solutions.comLinkedIn: linkedin.com/ in/akanksha-s-498328214w
ww.zodiac-solutions.com
Not the right fit? Search for Databricks Data Engineer jobs in Toronto, Ontario, Canada
About Zodiac Solutions, Inc
“Zodiac Solutions” is a “Minority Business Enterprise” headquartered in Texas, USA.
Our clients include large and small enterprises in the entertainment, media, financial, government, and healthcare sectors. We’ve carved out a niche as one of the industry’s top IT solutions providers by doing what we do best. To serve our diversified clientele, we go out of our way to learn about their business and assist them as needed.
Zodiac Solutions works on a client-centric approach guided by the business philosophy of building lasting customer relationships in an intensely competitive global environment.”
Similar Jobs
Principal Databricks Data Engineer
About the role
Job Title: Principal Databricks Data Engineer Location: Toronto, ON onsite Job Type: Fulltime Permanent Role
Job Summar
y **Key Requirements **12–18 years of overall data engineering experien ce8+ years of experience in enterprise Data Warehouse and Data Lake platfor ms5+ years of hands-on experience with Databricks and Spark at sca leStrong experience in modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehou seRedesign ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta La keRefactor legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelin esEnsure data reconciliation, audit integrity, and consistency during migrati onDesign and govern enterprise Data Warehouse and Data Lake/Lakehouse architectur esImplement layered architecture including Raw/Landing, Curated/Conformed, and Semantic/Consumption laye rsModernize traditional EDW platforms into scalable lakehouse architectur esStrong experience in finance and risk data models including General Ledger, Sub-ledger, financial hierarchies, and risk exposure models (credit, liquidity, market ris
k)
- Enable reporting use cases including aggregation, drill-down, and drill-back capabilit
ies
- Build and manage semantic/consumption layers for BI, reporting, and analy
tics
- Define business metrics, dimensions, hierarchies, and
KPIs
- Experience with Databricks SQL, Delta tables, and dbt or similar fram
eworks
- Develop and optimize large-scale data pipelines using PySpark, Spark SQL, and Del
ta Lake
- Implement Medallion architecture (Bronze, Silver, Gold
layers)
- Optimize workloads using Z-ORDER, OPTIMIZE, caching, and cluster confi
gurations
- Implement data governance, data quality frameworks, reconciliation controls, and exceptio
n handling
- Establish data lineage and metadata
management
- Ensure data security, access control, and complian
ce standards
- Experience with cloud platforms such as
AWS or Azure
- Experience with CI/CD pipelines using Git, Terraform, Jenkins, o
r Azure DevOps
- Familiarity with orchestration tools such as Airflow or Datab
ricks Workflows
- Experience wi
th dbt is a plus
- Act as a technical authority and lead archi
tecture decisions
- Mentor and guide senior engineers and establish eng
ineering standards
- Strong stakeholder management with finance, risk, analytics, a
nd governance teams
- Ability to translate complex data structures into bus
iness-ready insi
ghts Nice to Have
- Experience in BFSI, Capital Markets, o r regulatory reporting* Exposure to SAP Finance, Oracle Financials, or S/4HANA* Experience supp orting AI/ML workloads* Databricks o
r cloud certi fications Warm Regard s,Akanksha Singh BaghelSen ior Technical Recruite rPhone: +1-484-756-3700Email: akansha.sin gh@zodiac-solutions.comLinkedIn: linkedin.com/ in/akanksha-s-498328214w
ww.zodiac-solutions.com
Not the right fit? Search for Databricks Data Engineer jobs in Toronto, Ontario, Canada
About Zodiac Solutions, Inc
“Zodiac Solutions” is a “Minority Business Enterprise” headquartered in Texas, USA.
Our clients include large and small enterprises in the entertainment, media, financial, government, and healthcare sectors. We’ve carved out a niche as one of the industry’s top IT solutions providers by doing what we do best. To serve our diversified clientele, we go out of our way to learn about their business and assist them as needed.
Zodiac Solutions works on a client-centric approach guided by the business philosophy of building lasting customer relationships in an intensely competitive global environment.”