GCP Senior Data Engineer

Job summary

Design, build, and optimize batch and real-time data pipelines and scalable, cost-efficient GCP data platforms using tools such as Dataflow, Spark, Pub/Sub, and BigQuery. Lead and mentor a team of 10 data engineers, establish engineering and governance standards, optimize platform performance and costs, and collaborate on analytics and machine-learning solutions.

Job details

Technical Skills: Strong proficiency in SQL, Python, Java/Scala, Spark, and Apache Beam. GCP Expertise: Hands-on experience with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), Cloud Storage, and Terraform. Key Responsibilities Design, build, and optimize batch and real-time data pipelines. Develop ETL/ELT solutions using Dataflow, Spark, Pub/Sub, and BigQuery. Architect scalable and cost-efficient GCP data platforms. Lead and mentor a team of 10 data engineers. Establish coding, CI/CD, monitoring, data quality, and governance standards. Optimize performance, reliability, and GCP costs. Collaborate with architects, data scientists, and analysts on analytics/ML solutions. Nice to Have: GCP Professional Data Engineer certification, Vertex AI, real-time processing, dbt, and multi-cloud experience.

What you’ll do

Design, build, and optimize batch and real-time data pipelines and scalable, cost-efficient GCP data platforms using tools such as Dataflow, Spark, Pub/Sub, and BigQuery. Lead and mentor a team of 10 data engineers, establish engineering and governance standards, optimize platform performance and costs, and collaborate on analytics and machine-learning solutions.

Requirements

Requires strong proficiency in SQL, Python, Java or Scala, Spark, and Apache Beam, along with hands-on experience across GCP data services and Terraform. The role calls for senior-level data engineering capability; a GCP Professional Data Engineer certification, Vertex AI, dbt, real-time processing, and multi-cloud experience are nice to have.

Listed skills

• SQL · Preferred
• CI/CD · Preferred
• Java · Preferred
• Terraform · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• SQL
• Python
• Java
• Scala
• Spark
• Apache Beam
• BigQuery
• Dataflow
• Dataproc
• Pub/Sub
• Composer (Airflow)
• Cloud Storage
• Terraform
• ETL/ELT
• CI/CD
• Data Governance

Job areas

• Data & Analytics
• Technology
• Software
• Management & Leadership