Data Engineer Jobs in Vancouver, British Columbia, Canada
Create alert for “Data Engineer”
Vancouver, British Columbia, Canada
Data Engineer - Capital Markets
About the role
Key Responsibilities
- Capital Markets Data Modeling
- Design and maintain physical and logical data models for Capital Markets domains, including trade execution, portfolio positions, market data (ticks/pricing), and risk metrics.
- Translate complex financial hierarchies (e.g., fund-of-funds, multi-asset class structures) into optimized Databricksstructures.
- Ensure data models support temporal requirements, such as Point-in-Time (PIT) analysis and "As-Of" financial reporting.
- Advanced STM (Source-to-Target Mapping) Creation
- Lead the creation of comprehensive STM documents, detailing the journey from legacy financial systems and external providers (e.g., Bloomberg, Reuters, Aladdin) to ADLS Gen2.
- Define complex transformation logic, business rules, and data enrichment steps within the STM to guide Data Engineering squads.
- Map data lineage to ensure full traceability for regulatory compliance.
- Databricks & ADLS Performance Engineering
- Design Delta Lake structures that prioritize "Shuffle-free" joins for massive capital markets datasets.
- Implement optimized partitioning and Z-Ordering strategies specifically for time-series financial data to enable high-speed analytics.
- Utilize Unity Catalog to govern data access and maintain a centralized metadata repository for the GWAM ecosystem.
Required Skills & Qualifications
- Industry Expertise: 5+ years of experience in Capital Markets or Wealth Management, with a deep understanding of financial instruments and trade lifecycles.
- Technical Modeling: 7+ years of experience in data modeling, with a mastery of Azure Databricks and ADLS Gen2.
- STM Mastery: Proven track record of creating highly detailed Source-to-Target Mappings for complex data migration or integration projects.
- Data Engine Proficiency: Expert-level Spark SQL and PySpark. Ability to optimize data structures for the Spark Catalyst Optimizer.
- Storage Formats: Expertise in Delta Lake (ACID transactions, Time Travel) and Parquet optimization.
- Governance: Hands-on experience implementing data governance and security via Unity Catalog.
Technical Stack
- Compute: Azure Databricks (Jobs, SQL Warehouses).
- Storage: ADLS Gen2 (Delta/Parquet).
- Governance: Unity Catalog.
- Analysis Tools: SQL, Python, Excel (for data profiling).
- Documentation: Confluence/Visio for STM and ERDs.
Not the right fit? Search for Data Engineer jobs in Toronto, Ontario, Canada
About Dimensionless Technologies
Dimensionless Technologies is a global IT consultancy and AI services company that offers AI-based solutions to a diverse range of industries.
Founded in 2016 in Mumbai, India, our journey has been marked by a relentless pursuit of excellence and a commitment to innovation.
We are on a mission to transform businesses by harnessing the power of artificial intelligence. Our core values center around enabling organizations to enhance operational efficiency and achieve unprecedented growth.
We are driven by innovation, constantly pushing the boundaries of what's possible with AI. Our team has conceptualized, built, and deployed a spectrum of AI solutions that have enabled our clients to reach their business objectives. These solutions include security automation, fraud detection, and natural language processing, among others.
Dimensionless' team of experts includes dedicated data scientists, machine learning engineers, and software developers. Their deep domain knowledge and technical prowess enable us to craft bespoke AI solutions that precisely meet the unique needs of each client.
With a strong presence in key global locations, including the US and India, Dimensionless Technologies stands ready to serve a diverse clientele. We maintain offices in New Jersey and our hometown, Mumbai.