ETL Tester

Job summary

Support and optimize ETL/ELT pipelines and data ingestion and Data Lake processing using Azure Data Factory and Azure Databricks. Perform end-to-end testing, source-to-target validation, reconciliation, and data quality checks; troubleshoot pipeline issues and automate validation and regression activities.

Job details

ETL Test Automation Engineer Location : Toronto, Ontario We are looking for a highly motivated 6-8 years of ETL Test Automation Engineer with strong expertise in Azure Data Factory (ADF), Azure Databricks, Python, SQL, and ETL Testing. Key Responsibilities: • Support ETL/ELT pipelines using Azure Data Factory and Azure Databricks. • Optimize data ingestion frameworks and Data Lake processing pipelines. • Implement data quality checks, data reconciliation, and validation frameworks. • Monitor and troubleshoot data pipeline failures, performance bottlenecks, and production issues. • Perform end-to-end ETL testing across source and target systems. • Execute source-to-target validation, reconciliation, schema validation, and data integrity checks. • Create and execute SQL-based validation scripts for large datasets. • Validate transformation logic, business rules, and incremental data loads. • Identify, track, and support closure of ETL and data quality defects. • Support SIT, UAT, Regression, and Production validation activities. • Automate regression and reconciliation activities to reduce manual effort. • Integrate automated validation scripts into CI/CD pipelines. • Drive continuous improvement through automation and AI-assisted testing practices. Required Technical Skills: • Azure Databricks • Azure Data Factory (ADF) • Python • PySpark • SQL / Azure SQL • ETL Testing • Bitbucket / Git

What you’ll do

Support and optimize ETL/ELT pipelines and data ingestion and Data Lake processing using Azure Data Factory and Azure Databricks. Perform end-to-end testing, source-to-target validation, reconciliation, and data quality checks; troubleshoot pipeline issues and automate validation and regression activities.

Requirements

The role seeks a mid-senior ETL Test Automation Engineer with 6–8 years of experience and expertise in Azure Data Factory, Azure Databricks, Python, PySpark, SQL, and ETL testing. Candidates should be able to validate transformations and data integrity, create SQL-based checks, support testing and production validation, and work with Git or Bitbucket.

Listed skills

• SQL · Preferred
• CI/CD · Preferred
• Data Validation · Preferred
• Git · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Azure Databricks
• Azure Data Factory
• Python
• PySpark
• SQL
• Azure SQL
• ETL Testing
• Git
• Bitbucket
• Data Validation
• Data Reconciliation
• Data Quality
• CI/CD
• Regression Testing
• Data Lake Processing

Job areas

• Technology
• Data & Analytics
• Software

More jobs from Vaspire Technologies Inc.