Data Engineer (Canadian Citizen or PR)
Design and implement data pipelines for structured and unstructured data, focusing on semantic consistency and deduplication. Develop and optimize search infrastructure and real-time embedding updates for RAG and agent memory systems.
- On-site
- Brampton, ON
- Posted Aug 4, 2026
- Apply by Sep 3, 2026
- 1 position
Job summary
Skillset Requirements • ETL & Data Modeling: Designing pipelines for structured/unstructured data, normalization, deduplication, and semantic consistency. • Vector Databases: pgvector, Redis, Azure AI Search, hybrid search, and index optimization. • Distributed Data Systems: Kafka, Spark, Flink, or similar event-driven architectures. • Data Governance: Zero-trust access, privacy controls, compliance, and auditability. • Real-time Embedding Updates: Event-driven refresh pipelines for RAG and agent memory systems. • Chunking & Embeddings: Semantic chunking, metadata tagging, and embedding model selection. • Search Infrastructure: BM25, hybrid search, inverted indexes, and ranking algorithms. • Performance Tuning: High-throughput read/write optimization. • Data Quality & Lineage: Validation, schema enforcement, and lineage tracking (e.g., Great Expectations, OpenLineage).
What you’ll do
Design and implement data pipelines for structured and unstructured data, focusing on semantic consistency and deduplication. Develop and optimize search infrastructure and real-time embedding updates for RAG and agent memory systems.
Requirements
Requires expertise in distributed data systems, vector databases, and data governance. Candidates must be proficient in performance tuning, chunking strategies, and data lineage tracking.
Listed skills
- RedisPreferred
Other relevant skills
Identified from the job description. Confirm important requirements above.
- ETL
- Data Modeling
- Vector Databases
- Pgvector
- Redis
- Azure AI Search
- Kafka
- Spark
- Flink
- Data Governance
- RAG
- Semantic Chunking
- BM25
- Performance Tuning
- Data Quality
- Lineage Tracking
Job areas
- Data & Analytics
- Technology
- Software
- Consulting
- Engineering
Additional details
- Minimum experience
- 5+ years
- Apply by
- Sep 3, 2026
- Posting language
- English
- Working hours
- 40 hours per week
- Seniority
- Mid-Senior level
- Application method
- Direct apply is available
