Data Test Engineer

Job summary

Build and validate data solutions, pipelines, and ETL/ELT processes using Databricks, Python, PySpark, and Azure. Own automated testing and end-to-end quality assurance across data pipelines, APIs, and backend systems.

Job details

Role Overview We are seeking a Senior Data Engineer with strong Quality Engineering (SDET) experience. The role is primarily focused on building and validating data solutions using Databricks, Python, and Azure cloud technologies, while also owning automated testing, end-to-end validation, and quality assurance across data pipelines, APIs, and backend systems. Key Responsibilities & Requirements: 5+ years of experience in Data Engineering, SDET, or Quality Engineering. Strong hands-on experience with Databricks and PySpark. Develop and validate data pipelines and ETL/ELT processes. Implement data quality and data validation practices. Perform REST API test automation using Karate or similar tools. Strong expertise in end-to-end testing. Experience with Microsoft Azure. Knowledge of Kafka and asynchronous backend testing. Experience with Databricks testing and data validation frameworks. Nice to Have: Docker and Kubernetes. E-commerce domain experience. Git, GitHub, and Jira.

What you’ll do

Build and validate data solutions, pipelines, and ETL/ELT processes using Databricks, Python, PySpark, and Azure. Own automated testing and end-to-end quality assurance across data pipelines, APIs, and backend systems.

Requirements

Requires 5+ years of experience in data engineering, SDET, or quality engineering, with hands-on Databricks and PySpark experience and expertise in end-to-end testing. Candidates should have experience with Azure, REST API test automation, data validation frameworks, and Kafka or asynchronous backend testing; Docker, Kubernetes, and e-commerce experience are nice to have.

Listed skills

• Python · Preferred
• Microsoft Azure · Preferred
• Docker · Preferred
• Kubernetes · Preferred
• Git · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Data Engineering
• Quality Engineering
• Databricks
• PySpark
• Python
• Azure
• Data Pipelines
• ETL/ELT
• Data Quality
• REST API Testing
• Karate
• End-to-End Testing
• Kafka
• Docker
• Kubernetes
• Git

Job areas

• Data & Analytics
• Technology
• Software
• Engineering

More jobs from TechDoQuest