Data Engineer

Job summary

Build and maintain data engineering solutions, including streaming ingestion pipelines, using Databricks, Unity Catalog, Scala, and AWS services. Work with LLMs and agent-based systems while supporting data governance, protection, retention, and lifecycle requirements.

Job details

Role: Data Engineer Location: Remote Canada Job Type : Contract Mandatory Skills: Databricks and data engineering, streaming data ingestion stream pipelines and Unity Catalog. Scala AWS (s3, Kinesis and Lambda) Terraform Other skills: Working with LLMs, particularly Claude. Creating MCPs, Skills and agents. Agents swarm and harness. Data governance and protection: PII, PCI, data retention and lifecycle. Domain exposure would be a plus: Subscriptions Billing, invoicing and payments.

What you’ll do

Build and maintain data engineering solutions, including streaming ingestion pipelines, using Databricks, Unity Catalog, Scala, and AWS services. Work with LLMs and agent-based systems while supporting data governance, protection, retention, and lifecycle requirements.

Requirements

The role requires experience with Databricks, data engineering, streaming pipelines, Unity Catalog, Scala, AWS services (S3, Kinesis, and Lambda), and Terraform. Experience with LLMs, Claude, MCPs, skills, agents, and data governance and protection is also relevant; subscription billing, invoicing, or payments domain exposure is a plus.

Listed skills

• Amazon Web Services · Preferred
• Terraform · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Databricks
• Data Engineering
• Streaming Data Ingestion
• Stream Pipelines
• Unity Catalog
• Scala
• AWS
• Amazon S3
• Amazon Kinesis
• AWS Lambda
• Terraform
• Large Language Models
• Model Context Protocol
• AI Agents
• Data Governance
• Data Protection

Job areas

• Technology
• Data & Analytics
• Software
• Engineering