Senior Machine Learning Systems Engineer – LLM Inference Performance on AWS Trainium

Job summary

Optimize LLM inference latency, throughput, and serving costs on AWS Trainium, including speculative decoding, long-context workloads, multi-chip scaling, and support for MoE and hybrid state-space models. Profile and optimize performance across frameworks, compilers, and kernels, create reproducible benchmarks, and provide customer teams with technical documentation and implementation guidance.

Job details

We’re Hiring: Senior Machine Learning Systems Engineer – LLM Inference Performance on AWS Trainium Location: Canada (Remote) Aptino Inc. is looking for an experienced Senior Machine Learning Systems Engineer to optimize large language model (LLM) inference performance across AWS Trainium platforms, including Trainium2 and Trainium3. This is a hands-on, customer-facing opportunity for an engineer passionate about LLM serving, accelerator optimization, performance benchmarking, and production-scale AI systems. Key Responsibilities: Optimize LLM decode latency, throughput, and serving costs on AWS Trainium. Implement and evaluate speculative decoding and Multi-Token Prediction (MTP) for Qwen-family models using vLLM on AWS Neuron. Design reproducible performance benchmarks comparing Trainium and leading GPU platforms. Optimize long-context inference workloads (100K+ tokens) and multi-chip scaling. Extend inference support for Mixture-of-Experts (MoE) and hybrid state-space models. Profile and optimize workloads at the framework, compiler, and kernel levels. Develop technical documentation, runbooks, and implementation guides to enable customer teams. Required Skills: ✅ 5+ years of experience in machine learning or systems engineering, including 2+ years in production ML inference or serving. ✅ Strong Python and PyTorch skills. ✅ Hands-on experience with vLLM, TensorRT-LLM, SGLang, or Triton Inference Server. ✅ Strong understanding of LLM prefill/decode, continuous batching, KV-cache management, and tensor/expert parallelism. ✅ Experience with accelerator profiling, performance optimization, and rigorous benchmarking. ✅ Excellent technical communication and problem-solving skills. Preferred Qualifications: AWS Trainium/Inferentia and AWS Neuron SDK experience. Experience with speculative decoding, Medusa, EAGLE, or MTP. Kernel development using NKI, Triton, or CUDA. Experience with Mamba, MoE architectures, FP8/INT8 quantization, and model quality validation. Customer-facing engineering, consulting, or technical enablement experience. 📩 Interested or know someone who would be a great fit? Please share your updated resume or reach out directly. #Hiring #MachineLearning #LLM #GenerativeAI #AWS #AWSNeuron #AWSTrainium #PyTorch #vLLM #AIJobs #RemoteJobs #CanadaJobs #ContractJobs #MLEngineering

What you’ll do

Optimize LLM inference latency, throughput, and serving costs on AWS Trainium, including speculative decoding, long-context workloads, multi-chip scaling, and support for MoE and hybrid state-space models. Profile and optimize performance across frameworks, compilers, and kernels, create reproducible benchmarks, and provide customer teams with technical documentation and implementation guidance.

Requirements

Requires at least five years of machine learning or systems engineering experience, including two or more years in production ML inference or serving, along with strong Python and PyTorch skills. Candidates should have hands-on experience with an inference-serving platform, knowledge of LLM serving concepts and parallelism, and experience profiling accelerators and conducting rigorous benchmarks; AWS Neuron, Trainium, and related techniques are preferred.

Listed skills

• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Python
• PyTorch
• LLM Inference
• vLLM
• AWS Neuron
• AWS Trainium
• Performance Optimization
• Performance Benchmarking
• Speculative Decoding
• Multi-Token Prediction
• KV-Cache Management
• Continuous Batching
• Tensor Parallelism
• Expert Parallelism
• Accelerator Profiling
• Kernel Development

Job areas

• Technology
• Software
• Engineering
• Data & Analytics
• Consulting

More jobs from Aptino, Inc.