Sr. Data Test Engineer (PySpark Databricks)

Job summary

Build and validate data solutions, pipelines, and ETL/ELT processes using Databricks, PySpark, Python, and Azure. Own automated testing and end-to-end quality assurance across data pipelines, APIs, and backend systems, including data validation and asynchronous testing.

Job details

Position Name – Sr. Data Test Engineer Type of hiring – Fulltime Location – Remote Canada Job Description: We are seeking a Senior Data Engineer with strong Quality Engineering (SDET) experience. The role is primarily focused on building and validating data solutions using Databricks, Python, and Azure cloud technologies, while also owning automated testing, end-to-end validation, and quality assurance across data pipelines, APIs, and backend systems. Key Responsibilities & Requirements: 5+ Years of experience in Data Engineering, SDET, or Quality Engineering. Strong hands-on experience with Databricks and PySpark. Develop and validate data pipelines and ETL/ELT processes. Implement data quality and data validation practices. Perform REST API test automation using Karate or similar tools. Strong expertise in end-to-end testing. Experience with Microsoft Azure. Knowledge of Kafka and asynchronous backend testing. Experience with Databricks testing and data validation frameworks. Nice to Have: Docker and Kubernetes. E-commerce domain experience. Git, GitHub, and Jira.

What you’ll do

Build and validate data solutions, pipelines, and ETL/ELT processes using Databricks, PySpark, Python, and Azure. Own automated testing and end-to-end quality assurance across data pipelines, APIs, and backend systems, including data validation and asynchronous testing.

Requirements

Requires 5+ years of experience in data engineering, SDET, or quality engineering, with hands-on Databricks and PySpark experience and expertise in end-to-end testing. Candidates should have experience with Azure, REST API test automation using Karate or similar tools, Kafka, and data validation frameworks; Docker, Kubernetes, e-commerce, Git, GitHub, and Jira are nice to have.

Listed skills

• Microsoft Azure · Preferred
• Automated testing · Preferred
• Data Validation · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Databricks
• PySpark
• Python
• Azure
• Data Engineering
• Quality Engineering
• Automated Testing
• Data Pipelines
• ETL/ELT
• Data Validation
• REST API Testing
• End-to-End Testing
• Karate
• Kafka
• Asynchronous Backend Testing
• Data Quality

Job areas

• Technology
• Data & Analytics
• Software
• Engineering

More jobs from Yochana