About
Senior Azure Data Engineer with 14+ years of experience architecting and delivering cloud-native data platforms across banking, financial services, and enterprise environments. Specialized in Microsoft Azure and Microsoft Fabric, including Azure Data Factory, Fabric Data Factory, Azure Databricks, Azure Synapse Analytics, OneLake, Delta Lake, and Lakehouse Architecture. Expertise in building scalable Medallion-based Lakehouse solutions, metadata-driven ETL frameworks, enterprise data warehouses, and streaming data pipelines. Proven track record of implementing CDC, SCD Type 2, data quality frameworks, DataOps practices, CI/CD automation, and Azure DevOps-based release management to support secure, governed, and production-grade analytics platforms.
Skills
- Python
- SQL
Experience
Integration Analyst
BMO Financial Group (Contractor via TCS)
May 2021 to Present
Toronto, Canada
• Architected and deployed end-to-end cloud data engineering pipelines using Azure Data Factory, Microsoft Fabric Data Factory, Azure Databricks, PySpark, Azure Data Lake Storage, and OneLake, processing over 100 million records daily. • Designed and implemented scalable Lakehouse architecture using Delta Lake across Azure Data Lake Storage and Microsoft Fabric OneLake following the Bronze-Silver-Gold (Medallion) architecture, enabling real-time analytics and historical reporting. • Developed Delta Lake tables within Azure Databricks and Microsoft Fabric Lakehouse using ACID transactions, schema enforcement, schema evolution, and Time Travel for reliable analytical workloads. • Experience with Git-based source control using Azure Repos, including branching, pull requests, merge conflict resolution, version control, and collaborative code reviews. • Developed analytical solutions using Azure Synapse Analytics (Dedicated and Serverless SQL Pools) and Microsoft Fabric SQL Endpoint to support enterprise reporting and analytical workloads. • Implemented Delta Live Tables (DLT) in Azure Databricks for building reliable, maintainable data pipelines with automated data quality checks. • Created dynamic, parameterized Azure Data Factory pipelines supporting 10+ diverse data sources (SQL Server, Oracle, JSON, XML, Excel). • Implemented incremental and full-load ingestion strategies using Azure Data Factory and Fabric Data Factory with robust error handling, monitoring, and alerting through Logic Apps. • Designed near real-time data pipelines using Azure Databricks Structured Streaming to process employee badge access events and support operational occupancy monitoring across corporate office locations. • Implemented event-driven data processing solutions using Azure Functions integrated with Azure Event Hubs and Azure Data Factory to support scalable and automated data ingestion workflows. • Implemented Change Data Capture (CDC) to track real-time data changes and enabled incremental processing across enterprise systems. • Implemented DataOps and CI/CD frameworks using Azure DevOps, Git, ARM Templates, DACPAC deployments, and automated release pipelines to support enterprise-scale Azure Data Lakehouse solutions across multiple environments. • Designed and implemented Slowly Changing Dimension (SCD Type 2) logic to maintain historical data tracking and auditability in data warehouse models. • Validated Azure Data Factory pipelines and Mapping Data Flows using the ADF Debugger, ensuring accurate data transformations prior to deployment. • Configured Self-Hosted Integration Runtime (SHIR), managed linked services, and optimized connectivity for enterprise ETL workloads. • Experience implementing Azure RBAC, Entra ID security groups, managed identities, service principals, and least-privilege access models across Azure and Microsoft Fabric environments. • Developed PySpark notebooks and SQL-based transformations within Microsoft Fabric Lakehouse to perform data cleansing, enrichment, aggregation, and preparation for downstream analytics and Power BI reporting. • Participated in Agile delivery teams using iterative development approaches similar to RAD methodology, delivering incremental solutions through rapid feedback cycles, prototype validation, sprint-based development, and continuous stakeholder engagement. • Managed Agile development using Azure Boards for sprint planning, backlog grooming, user stories, tasks, bugs, and work item tracking.
Data Architect
CPP (Contractor via TCS)
Nov 2019 to Apr 2021
Toronto, Canada
• Architected and optimized data integration workflows in Azure Data Factory to ingest high-volume data from SQL Server. • Implemented incremental data loading strategies using watermark columns (LastModifiedDate) to reduce data transfer costs. • Translated business requirements into technical solutions by designing ETL/ELT workflows, data transformation logic, source-to-target mappings, and scalable data processing frameworks using Azure data services. • Created Mapping Data Flows for complex data transformations and cleansing within Azure Data Factory.
Senior Developer
FCC (Contractor via TCS)
Oct 2016 to Oct 2019
Regina, Saskatchewan
• Developed enterprise ETL solutions using Informatica PowerCenter, processing 50+ million records daily with 99.9% reliability. • Implemented SCD Type 1 and SCD Type 2 logic for historical and current data tracking in data warehouse solutions. • Worked with business teams to understand reporting requirements and developed data models supporting enterprise analytics and reporting solutions using IBM Cognos Analytics. • Loaded data from Oracle to Teradata using Teradata TPT, handling complex transformations and data validation. • Designed source-to-target mappings, transformation rules, and data processing logic for integrating property, lease, and operational datasets into Azure cloud data platforms. • Participated in Agile Scrum ceremonies including sprint planning, backlog refinement, daily standups, sprint reviews, and retrospectives.
Senior Technical Consultant
CGI
Jan 2015 to Oct 2016
Halifax, Canada
• Developed and deployed ETL solutions using Informatica PowerCenter (Designer, Workflow Manager, Repository Manager, Workflow Monitor), replacing legacy Oracle PL/SQL processes and supporting enterprise data integration. • Conducted code reviews and mentored junior developers in ETL best practices and performance optimization techniques. • Performed SQL and ETL performance tuning, reducing batch processing time by 35%.
Senior Technical Consultant
Teradata India Pvt. Ltd
Nov 2012 to Dec 2014
Mumbai, India
• Developed Informatica workflows and reusable mapplets for enterprise data integration projects. • Performed SQL and ETL performance tuning across production systems. • Automated batch processes using shell scripting to improve operational efficiency.