Who you are

• Streaming experience preferred
• 6+ years of data engineering experience in big data environments
• Proven experience designing and implementing streaming architectures
• Extensive hands-on DBT experience (models, macros, tests, documentation)
• Strong Amazon Redshift architecture and performance optimization expertise
• Experience building CI/CD pipelines for data platforms
• Experience working in client-facing delivery contexts
• AWS: Redshift, S3, Glue, Step Functions, Lambda (Python), Athena, EMR
• Strong SQL and Redshift performance tuning expertise
• Python and PySpark (or equivalent distributed processing frameworks)
• Git-based version control workflows
• Deep understanding of data warehousing, modeling, and big data systems

What the job involves

• 12 Month contract
• We are seeking a Lead Data Engineer to join a client-focused data pod delivering large-scale data engineering solutions within a cloud-native AWS environment
• This is a delivery-first role requiring deep hands-on expertise in streaming data architectures, big data systems, and modern data warehousing practices. You will own streaming architectural direction while remaining actively involved in implementation
• The environment is AWS-centric (Redshift, S3, Glue, Step Functions, Lambda, EMR), with DBT as the transformation framework. We are actively integrating streaming data from GCP sources into our AWS data platform
• You will define engineering standards across data modeling, DBT implementation, testing, CI/CD, and production resiliency while collaborating directly with the client’s data team
• Own the architectural direction for streaming data ingestion from GCP into AWS
• Design resilient ingestion frameworks including error handling, retry strategies, monitoring, and failure isolation
• Implement distributed processing pipelines using Spark / PySpark or similar frameworks
• Create and maintain scalable data warehouses and associated ETL/ELT processes using DBT models in Amazon Redshift
• Design and implement DBT projects including macros, tests, documentation, and reusable modeling patterns
• Conduct Redshift query and DBT performance tuning to optimize warehouse efficiency and cost
• Define and enforce best practices for:
• Data modeling
• Version control (Git-based workflows)
• CI/CD pipelines for DBT deployments
• Automated testing at model, transformation, and pipeline levels
• Ensure robust testing is embedded into every DBT model (schema tests, custom tests, data validation checks)
• Lead code reviews and architectural design reviews
• Work with AWS services including Redshift, S3, Glue, Step Functions, Lambda (Python), Athena, and EMR

Benefits

• Renowned Team Days often throughout the year
• Summer Away Days
• 23 standard days of holiday (+ 1 Birthday, +2 for a ‘Big Life Event’ and +1 Admin Day), closure of office over Christmas (plus Bank Holidays)
• Discretionary Bonus based on company performance
• Performance Reviews every 6 months with discretionary salary increases
• Private healthcare (Vitality) and/or Health Care Plan (Medicash)
• Mobile phone contribution
• Sport Challenge contribution
• Gym membership contribution
• 2x annual kit drops
• Tickets to sporting events
• Lunch once a week, breakfast and continuous supply of snacks
• Private healthcare schemes
• Cycle to work scheme
• Learning and Development opportunities, including certification in certain areas