Machine Learning Engineer — Inference Optimization
Featherless AI · Remote (world) · Posted Jan 22, 2026
Transparency
DeadOnline for 43 weeks and never reposted. Jobs this old or reposted this often rarely lead to a hire. It may be filled already.
- First seen
- Oct 10, 2026
- Offered for
- 302 days
Overview
- Category
- AI & machine learning
- Employment type
- Permanent
- Workplace type
- Remote
- Country
- United States
- Seniority
- Senior
- Schedule
- Full time
Key skills
- Inference Performance Optimization
- Improve latency, throughput, and cost of large‑scale ML models in production.
- GPU/CPU Profiling
- Profile and remove bottlenecks in kernel, memory, batching, and I/O pipelines.
- Quantization & Pruning
- Apply fp16/bf16/int8/fp8 quantization and model pruning for faster inference.
- Inference Serving Systems
- Build and maintain serving stacks such as Triton or custom runtimes.
- PyTorch Model Deployment
- Deploy and tune deep‑learning models using PyTorch or equivalent frameworks.
- Hardware Benchmarking
- Benchmark inference across NVIDIA, AMD GPUs and CPUs in cloud environments.
Key phrases
- model inference performance at scale
- profiling systems down to the kernel/GPU level
- throughput, and cost
- GPU/CPU inference pipelines
- Quantization (fp16, bf16, int8, fp8)
- Speculative decoding, batching, and streaming
- Model pruning
- productionize new model architectures
- Triton, custom runtimes, or bespoke stacks
- Benchmark performance across hardware
- TensorRT, ONNX Runtime, vLLM
From the posting
About the Role
We’re looking for a Machine Learning Engineer to own and push the limits of model inference performance at scale. You’ll work at the intersection of research and production—turning cutting-edge models into fast, reliable, and cost-efficient systems that serve real users.
This role is ideal for someone who enjoys deep technical work, profiling systems down to the kernel/GPU level, and translating research ideas into production-grade performance gains.
What You’ll Do
Optimize inference latency, throughput, and cost for large-scale ML models in production
Profile and bottleneck GPU/CPU inference pipelines (memory, kernels, batching, IO)
Implement and tune techniques such as:
Quantization (fp16, bf16, int8, fp8)
KV-cache optimization & reuse
Speculative decoding, batching, and streaming
Model pruning or architectural simplifications for inference
Collaborate with research engineers to productionize new model architectures
Build and maintain inference-serving systems (e.g. Triton, custom runtimes, or bespoke stacks)
Benchmark performance across hardware (NVIDIA / AMD GPUs, CPUs) and cloud setups
Improve system reliability, observability, and cost efficiency under real workloads
What We’re Looking For
Strong experience in ML inference optimization or high-performance ML systems
Solid understanding of deep learning internals (attention, memory layout, compute graphs)
Hands-on experience with PyTorch (or similar) and model deployment
Familiarity with GPU performance tuning (CUDA, ROCm, Triton, or kernel-level optimizations)
Experience scaling inference for real users (not just research benchmarks)
Comfortable working in fast-moving startup environments with ownership and ambiguity
Nice to Have
Experience with LLM or long-context model inference
Knowledge of inference frameworks (TensorRT, ONNX Runtime, vLLM, Triton)
Experience optimizing across different hardware vendors
Open-source contributions in ML systems or inference tooling
Background in distributed systems or low-latency services
Why Join Us
Real ownership over performance-critical systems
Direct impact on product reliability and unit economics
Close collaboration with research, infra, and product
Competitive compensation + meaningful equity at Series A
A team that cares about engineering quality, not hype
