Staff Artificial Intelligence Engineer
Top Benefits
About the role
Who you are
- Ideally 7+ years of software engineering experience, with 3+ years building ML/AI-powered features in production
- Proven experience shipping production AI/GenAI features end-to-end
- Strong background as a Staff-level product software engineer (not data science or ML-research-only)
- Experience iteratively improving GenAI-powered features based on quantitative and qualitative signals
- Ability to design, run, and evaluate experiments for AI features
- Prior on-call experience supporting production systems
- Deep expertise in LLMs, prompt engineering, retrieval-augmented generation, and evaluation methods
- Proficiency in backend development using Python, Go, Ruby, TypeScript or a similar language
- Strong cross-functional skills and a track record of collaborating closely with PMs and designers
- Comfort navigating ambiguity and driving clarity in fast-paced environments
- Bonus: Experience with observability tools, production infrastructure, or public contributions to the AI community
What the job involves
- We’re scaling our dedicated AI Team—a group of builders shaping the future of developer tools with AI at the core
- This isn’t just about building cool demos—it’s about shipping reliable, innovative AI-powered features that make our users faster, smarter, and more effective
- As a Staff AI Engineer, you’ll lead high-impact projects, mentor teammates, and help define what great AI at Honeycomb looks like
- We’ve already shipped major AI capabilities like an AI-guided workspace to query your Honeycomb data, Natural Language Querying, and an MCP server, and shared our learnings along the way in posts like So we shipped an AI Product, did it work?
- The AI team is dedicated to building user-facing AI features that transform how engineers explore and understand their software systems
- Along with the rest of the team’s triad, you’ll help lead a cross-functional team reshaping Honeycomb into an AI-native platform for observability
- If you’re excited about owning high-impact, greenfield projects and helping define what “great AI” looks like in production, we’d love to meet you
- Own end-to-end implementation of AI-powered product features, from prototypes to production. Mentor other engineers on the team, leveling up the team as a whole. Collaborate across the organization to support shipping these features to production
- Handle prompt engineering, model tuning, and evaluation strategies to improve reliability and UX
- Collaborate with product and design partners in defining and scoping AI-driven experiences
- Teach best practices for working with LLMs, including prompt design and evaluation methodologies
- Learn deeply about Honeycomb’s workflows and user problems to build truly helpful AI features
- Iterate through fast feedback loops, experimentation, and sharing both wins and failures
Benefits
- 100% covered medical, dental, vision, HSA, FSA, disability, and more
- Unlimited PTO, synchronous company break, paid sabbatical, and expressed minimum for time to recharge
- 401K retirement plans and fully paid life insurance
- Up to 16 weeks paid leave regardless of path to parenthood; flexible hours for returning parents and caregivers
- $10K to cover gaps in insurance, in support of adoption, surrogacy, preservation, and family-planning costs
- $600 per year to spend on the perks you care for most
- A stake in our success
- Generous new hire equity, and refresh grants
- Annual $1,500 development stipend; funding for cognitive coaching; and a focus on career pathing
- $500 new hire home setup stipend and $200/mo for wifi, phone and co-working
About honeycomb.io
Honeycomb is the observability platform that enables engineering teams to find and solve problems they couldn't before. Honeycomb’s approach is fundamentally different from other tools that claim observability, and is built to help teams answer novel questions about their ever-evolving cloud applications.
Other tools silo your data across disjointed pillars (logs, metrics, and traces), are too slow, and constrain teams to only answer predetermined questions. Honeycomb unifies all data sources in a single type, returning queries in seconds – not minutes – and revealing critical issues that logs and metrics alone can’t see. Using the power of distributed tracing and a query engine designed for highly-contextual telemetry data, Honeycomb reveals both why a problem is happening and who specifically is impacted.
Every interface is interactive, enabling any engineer – no matter how tenured – to ask questions on the fly, drill down by any dimension and solve issues before customers notice. Powered by machine learning, BubbleUp enables you to quickly spot outliers that reveal the cause of hidden issues. Engineering teams can deploy confidently, resolve incidents faster and focus on high-value work that drives innovation.
Staff Artificial Intelligence Engineer
Top Benefits
About the role
Who you are
- Ideally 7+ years of software engineering experience, with 3+ years building ML/AI-powered features in production
- Proven experience shipping production AI/GenAI features end-to-end
- Strong background as a Staff-level product software engineer (not data science or ML-research-only)
- Experience iteratively improving GenAI-powered features based on quantitative and qualitative signals
- Ability to design, run, and evaluate experiments for AI features
- Prior on-call experience supporting production systems
- Deep expertise in LLMs, prompt engineering, retrieval-augmented generation, and evaluation methods
- Proficiency in backend development using Python, Go, Ruby, TypeScript or a similar language
- Strong cross-functional skills and a track record of collaborating closely with PMs and designers
- Comfort navigating ambiguity and driving clarity in fast-paced environments
- Bonus: Experience with observability tools, production infrastructure, or public contributions to the AI community
What the job involves
- We’re scaling our dedicated AI Team—a group of builders shaping the future of developer tools with AI at the core
- This isn’t just about building cool demos—it’s about shipping reliable, innovative AI-powered features that make our users faster, smarter, and more effective
- As a Staff AI Engineer, you’ll lead high-impact projects, mentor teammates, and help define what great AI at Honeycomb looks like
- We’ve already shipped major AI capabilities like an AI-guided workspace to query your Honeycomb data, Natural Language Querying, and an MCP server, and shared our learnings along the way in posts like So we shipped an AI Product, did it work?
- The AI team is dedicated to building user-facing AI features that transform how engineers explore and understand their software systems
- Along with the rest of the team’s triad, you’ll help lead a cross-functional team reshaping Honeycomb into an AI-native platform for observability
- If you’re excited about owning high-impact, greenfield projects and helping define what “great AI” looks like in production, we’d love to meet you
- Own end-to-end implementation of AI-powered product features, from prototypes to production. Mentor other engineers on the team, leveling up the team as a whole. Collaborate across the organization to support shipping these features to production
- Handle prompt engineering, model tuning, and evaluation strategies to improve reliability and UX
- Collaborate with product and design partners in defining and scoping AI-driven experiences
- Teach best practices for working with LLMs, including prompt design and evaluation methodologies
- Learn deeply about Honeycomb’s workflows and user problems to build truly helpful AI features
- Iterate through fast feedback loops, experimentation, and sharing both wins and failures
Benefits
- 100% covered medical, dental, vision, HSA, FSA, disability, and more
- Unlimited PTO, synchronous company break, paid sabbatical, and expressed minimum for time to recharge
- 401K retirement plans and fully paid life insurance
- Up to 16 weeks paid leave regardless of path to parenthood; flexible hours for returning parents and caregivers
- $10K to cover gaps in insurance, in support of adoption, surrogacy, preservation, and family-planning costs
- $600 per year to spend on the perks you care for most
- A stake in our success
- Generous new hire equity, and refresh grants
- Annual $1,500 development stipend; funding for cognitive coaching; and a focus on career pathing
- $500 new hire home setup stipend and $200/mo for wifi, phone and co-working
About honeycomb.io
Honeycomb is the observability platform that enables engineering teams to find and solve problems they couldn't before. Honeycomb’s approach is fundamentally different from other tools that claim observability, and is built to help teams answer novel questions about their ever-evolving cloud applications.
Other tools silo your data across disjointed pillars (logs, metrics, and traces), are too slow, and constrain teams to only answer predetermined questions. Honeycomb unifies all data sources in a single type, returning queries in seconds – not minutes – and revealing critical issues that logs and metrics alone can’t see. Using the power of distributed tracing and a query engine designed for highly-contextual telemetry data, Honeycomb reveals both why a problem is happening and who specifically is impacted.
Every interface is interactive, enabling any engineer – no matter how tenured – to ask questions on the fly, drill down by any dimension and solve issues before customers notice. Powered by machine learning, BubbleUp enables you to quickly spot outliers that reveal the cause of hidden issues. Engineering teams can deploy confidently, resolve incidents faster and focus on high-value work that drives innovation.