Senior Data Engineer, (Global Security)
Top Benefits
About the role
What is the opportunity?
For a Senior Data Engineer to join our collaborative cyber security team, focusing on building scalable data solutions that directly enhance our application ecosystem. This role will work together with our application developers to integrate a robust data capability into our application to transform how our products leverage data for better design and business value. You’ll be instrumental in developing a comprehensive data strategy that seamlessly bridges the gaps between data and application functionality.
This role is focused on python application development with Databricks experience. Working directly with Python application developers to integrate data capabilities into existing and new applications.
What will you do?
- Design, develop, and maintain end-to-end data pipelines in Azure Databricks using Spark (SQL, PySpark).
- Implement and optimize ELT/ELT workflows using Databricks Workflows or Apache Airflow ensuring data integrity, quality, and reliability.
- Manage Delta Lake solutions for data versioning, incremental loads, and efficient application data access.
- Apply best practices in data governance, ensuring compliance using Unity Catalog for access management and data lineage tracking.
- Monitor, troubleshoot, and optimize Spark jobs for performance, addressing data pipelines bottlenecks that impact application responsiveness.
- Build automated monitoring, alerting, and incident managements solution to ensure data reliability, availability, and performance.
- Collaborate with our developers and cross-functional teams to integrate data capabilities into existing and new applications.
- Build APIs and data services that applications can consume for real-time and batch data processing.
- Develop and maintain comprehensive documentation for data pipelines, transformations and data models.
- Foster knowledge and collaborative functions
What do you need to succeed?
-
Databricks certifications (e.g., Databricks Certified Data Engineer, Spark Engineer).
-
Application Development experience, proficient in Python and building microservices.
-
Institutional knowledge working with an enterprise that embraces programming best practices
-
Bachelor’s or master’s degree in computer science, Data Engineering and, or a related field.
-
5+ years of proven experience in data engineering, delivering business-critical software solutions for large enterprises with a consistent track record of success.
-
Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Delta Live Pipelines, Databricks Runtime, Cluster management etc.)
-
Strong experience in Spark and PySpark for big data processing.
-
Knowledge of SCM, Infrastructure-as-code, and CI/CD pipelines.
Nice to Have
- Exposure to Kubernetes, Docker, and Terraform.
- Strong understanding of business intelligence and reporting tools.
- Familiarity with Cyber Security Concepts.
What’s in it for you?
We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice `to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.
- A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
- Leaders who support your development through coaching and managing opportunities
- Work in a dynamic, collaborative, progressive, and high-performing team
- A world-class training program in financial services
- Opportunities to do challenging work
- Opportunities to take on progressively greater accountabilities
- Opportunities to building close relationships with clients.
#LI-Post
#LI-PK
#TECHPJ
About RBC
Royal Bank of Canada is a global financial institution with a purpose-driven, principles-led approach to delivering leading performance. Our success comes from the 94,000+ employees who leverage their imaginations and insights to bring our vision, values and strategy to life so we can help our clients thrive and communities prosper. As Canada's biggest bank and one of the largest in the world, based on market capitalization, we have a diversified business model with a focus on innovation and providing exceptional experiences to our more than 17 million clients in Canada, the U.S. and 27 other countries. Learn more at rbc.com. We are proud to support a broad range of community initiatives through donations, community investments and employee volunteer activities. See how at www.rbc.com/community-social-impact.
La Banque Royale du Canada est une institution financière mondiale définie par sa raison d'être, guidée par des principes et orientée vers l'excellence en matière de rendement. Notre succès est attribuable aux quelque 94 000+ employés qui mettent à profit leur créativité et leur savoir faire pour concrétiser notre vision, nos valeurs et notre stratégie afin que nous puissions contribuer à la prospérité de nos clients et au dynamisme des collectivités. Selon la capitalisation boursière, nous sommes la plus importante banque du Canada et l'une des plus grandes banques du monde. Nous avons adopté un modèle d'affaires diversifié axé sur l'innovation et l'offre d'expériences exceptionnelles à nos plus de 17 millions de clients au Canada, aux États Unis et dans 27 autres pays. Pour en savoir plus, visitez le site rbc.com/francais
Nous sommes fiers d'appuyer une grande diversité d'initiatives communautaires par des dons, des investissements dans la collectivité et le travail bénévole de nos employés. Pour de plus amples renseignements, visitez le site www.rbc.com/collectivite-impact-social.
Senior Data Engineer, (Global Security)
Top Benefits
About the role
What is the opportunity?
For a Senior Data Engineer to join our collaborative cyber security team, focusing on building scalable data solutions that directly enhance our application ecosystem. This role will work together with our application developers to integrate a robust data capability into our application to transform how our products leverage data for better design and business value. You’ll be instrumental in developing a comprehensive data strategy that seamlessly bridges the gaps between data and application functionality.
This role is focused on python application development with Databricks experience. Working directly with Python application developers to integrate data capabilities into existing and new applications.
What will you do?
- Design, develop, and maintain end-to-end data pipelines in Azure Databricks using Spark (SQL, PySpark).
- Implement and optimize ELT/ELT workflows using Databricks Workflows or Apache Airflow ensuring data integrity, quality, and reliability.
- Manage Delta Lake solutions for data versioning, incremental loads, and efficient application data access.
- Apply best practices in data governance, ensuring compliance using Unity Catalog for access management and data lineage tracking.
- Monitor, troubleshoot, and optimize Spark jobs for performance, addressing data pipelines bottlenecks that impact application responsiveness.
- Build automated monitoring, alerting, and incident managements solution to ensure data reliability, availability, and performance.
- Collaborate with our developers and cross-functional teams to integrate data capabilities into existing and new applications.
- Build APIs and data services that applications can consume for real-time and batch data processing.
- Develop and maintain comprehensive documentation for data pipelines, transformations and data models.
- Foster knowledge and collaborative functions
What do you need to succeed?
-
Databricks certifications (e.g., Databricks Certified Data Engineer, Spark Engineer).
-
Application Development experience, proficient in Python and building microservices.
-
Institutional knowledge working with an enterprise that embraces programming best practices
-
Bachelor’s or master’s degree in computer science, Data Engineering and, or a related field.
-
5+ years of proven experience in data engineering, delivering business-critical software solutions for large enterprises with a consistent track record of success.
-
Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Delta Live Pipelines, Databricks Runtime, Cluster management etc.)
-
Strong experience in Spark and PySpark for big data processing.
-
Knowledge of SCM, Infrastructure-as-code, and CI/CD pipelines.
Nice to Have
- Exposure to Kubernetes, Docker, and Terraform.
- Strong understanding of business intelligence and reporting tools.
- Familiarity with Cyber Security Concepts.
What’s in it for you?
We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice `to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.
- A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
- Leaders who support your development through coaching and managing opportunities
- Work in a dynamic, collaborative, progressive, and high-performing team
- A world-class training program in financial services
- Opportunities to do challenging work
- Opportunities to take on progressively greater accountabilities
- Opportunities to building close relationships with clients.
#LI-Post
#LI-PK
#TECHPJ
About RBC
Royal Bank of Canada is a global financial institution with a purpose-driven, principles-led approach to delivering leading performance. Our success comes from the 94,000+ employees who leverage their imaginations and insights to bring our vision, values and strategy to life so we can help our clients thrive and communities prosper. As Canada's biggest bank and one of the largest in the world, based on market capitalization, we have a diversified business model with a focus on innovation and providing exceptional experiences to our more than 17 million clients in Canada, the U.S. and 27 other countries. Learn more at rbc.com. We are proud to support a broad range of community initiatives through donations, community investments and employee volunteer activities. See how at www.rbc.com/community-social-impact.
La Banque Royale du Canada est une institution financière mondiale définie par sa raison d'être, guidée par des principes et orientée vers l'excellence en matière de rendement. Notre succès est attribuable aux quelque 94 000+ employés qui mettent à profit leur créativité et leur savoir faire pour concrétiser notre vision, nos valeurs et notre stratégie afin que nous puissions contribuer à la prospérité de nos clients et au dynamisme des collectivités. Selon la capitalisation boursière, nous sommes la plus importante banque du Canada et l'une des plus grandes banques du monde. Nous avons adopté un modèle d'affaires diversifié axé sur l'innovation et l'offre d'expériences exceptionnelles à nos plus de 17 millions de clients au Canada, aux États Unis et dans 27 autres pays. Pour en savoir plus, visitez le site rbc.com/francais
Nous sommes fiers d'appuyer une grande diversité d'initiatives communautaires par des dons, des investissements dans la collectivité et le travail bénévole de nos employés. Pour de plus amples renseignements, visitez le site www.rbc.com/collectivite-impact-social.