Principal Observability Engineer

Job summary

Lead the architecture, design, and implementation of enterprise observability solutions, including dashboards, alerting, and monitoring across cloud and Kubernetes environments. Automate infrastructure, establish observability best practices, and provide technical leadership and mentorship to engineering teams.

Job details

Job title: Principal Observability Engineer Duration: 12-month contract (extension possible) Role status: Current opening Principal tasks and responsibilities include: Lead the architecture, design, and implementation of enterprise observability solutions and monitoring frameworks. Develop and enhance dashboards, alerting, and monitoring capabilities using Splunk, Splunk Observability, OpenTelemetry, Grafana, Prometheus, and AWS CloudWatch. Design and support observability across cloud and Kubernetes-based environments, driving platform performance and reliability. Develop infrastructure automation using Infrastructure as Code (IaC), scripting, and configuration management tools. Provide technical leadership, establish observability best practices, and mentor engineering teams. Our client: A large global enterprise organization Qualifications and pre-requisites: 10+ years of experience in Platform Engineering, Site Reliability Engineering (SRE), Systems Engineering, or a related infrastructure discipline. Strong hands-on experience with Splunk and modern observability platforms, including Splunk Observability, OpenTelemetry, Grafana, Prometheus, and AWS CloudWatch. Deep expertise with AWS, Kubernetes/OpenShift, and infrastructure automation tools such as Terraform, Ansible, Python, or Bash. Proven experience designing scalable monitoring and observability solutions within large enterprise environments. Strong communication and leadership skills with the ability to collaborate across technical and business teams. Additional information or perks: Experience with additional cloud platforms (Azure or GCP), Red Hat technologies, virtualization, storage, backup, or identity management is considered an asset. Relevant certifications such as Splunk, AWS, or Kubernetes are highly desirable. isgSearch does not use artificial intelligence throughout the hiring process.

What you’ll do

Lead the architecture, design, and implementation of enterprise observability solutions, including dashboards, alerting, and monitoring across cloud and Kubernetes environments. Automate infrastructure, establish observability best practices, and provide technical leadership and mentorship to engineering teams.

Requirements

Requires 10+ years of experience in platform, site reliability, systems, or related infrastructure engineering, with hands-on expertise in Splunk and modern observability platforms. Candidates should have deep AWS, Kubernetes or OpenShift, and infrastructure automation experience, along with strong communication and leadership skills; additional cloud, Red Hat, virtualization, storage, backup, identity management, and relevant certifications are assets.

Listed skills

• Kubernetes · Preferred
• Splunk · Preferred
• Amazon Web Services · Preferred
• Terraform · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Splunk
• Splunk Observability
• OpenTelemetry
• Grafana
• Prometheus
• AWS CloudWatch
• AWS
• Kubernetes
• OpenShift
• Infrastructure As Code
• Terraform
• Ansible
• Python
• Bash
• Platform Engineering
• Site Reliability Engineering

Job areas

• Technology
• Engineering
• Software
• Management & Leadership