Role Overview

Job Title: Sr Data Engineer Location: Mississauga / Hybrid Employment Type: Full-time Compensation: CAD 140K per annum(Including QPLC)(no deviations)

Job Summary

We are looking for a motivated and talented Sr Data Engineer to join our dynamic Capital Markets Data Engineering team. This role is perfect for an aspiring engineer who is passionate about building robust, scalable data solutions that power our financial analytics and AI initiatives. The ideal candidate will have a solid foundation in big data technologies, including advanced Spark, Python, and the Hadoop ecosystem, coupled with a strong interest in cloud computing. You will play a crucial role in developing and maintaining the data backbone that supports our trading, risk, and regulatory functions.

Key Responsibilities

· Data Pipeline Development: Design, build, and maintain efficient and reliable data pipelines using Python, Spark, and SQL to process large-scale financial datasets.

· Big Data Management: Work within our Hadoop and Cloudera environments to ingest, store, and manage data, ensuring its availability and performance for downstream applications.

· Performance Optimization: Write advanced, optimized Spark jobs and complex SQL queries to ensure data processing is performed with high efficiency and low latency.

· Cloud Integration: Assist in the deployment, management, and optimization of data engineering solutions on cloud platforms (e.g., AWS, GCP).

· Collaboration: Partner with data scientists, business analysts, and other technology teams to understand data requirements and deliver high-quality, integrated data solutions.

· Data Quality & Integrity: Implement data quality checks and validation processes to ensure the accuracy, completeness, and reliability of our data assets.

· Technical Support: Provide support for production data pipelines, troubleshooting issues and ensuring operational stability.

· Learning & Innovation: Continuously learn and apply new technologies and best practices in data engineering and cloud computing.

Required Qualifications & Skills

· Education: Bachelor’s degree in computer science, Engineering, Information Technology, or a related technical field.

· Programming Skills: Strong programming proficiency in Python and a solid understanding of software engineering principles.

· Big Data Foundation: Hands-on experience or strong academic knowledge of Apache Spark and the Hadoop ecosystem (HDFS, Hive, etc.).

· SQL Expertise: Proficiency in writing complex and efficient SQL queries for data manipulation and analysis.

· Cloud Knowledge: A good understanding of fundamental cloud computing concepts and services (e.g., object storage, compute instances, serverless functions). · Problem-Solving: Excellent analytical and problem-solving skills, with the ability to tackle complex technical challenges. · Team Player: Strong communication and interpersonal skills, with a collaborative mindset and a willingness to learn from senior engineers.

Preferred Qualifications · Experience with the Cloudera Data Platform (CDP). · Familiarity with cloud-native data warehousing and data lake technologies (e.g., Snowflake, Spark, Python, Hadoop, BigQuery, Databricks). · Knowledge of workflow orchestration tools like Apache Airflow. · Experience with containerization technologies such as Docker and Kubernetes. · Understanding CI/CD principles and tools (e.g., Jenkins, GitLab CI). · An interest in the financial services and capital markets domain.