Machine Learning Engineer — Inference Optimization

Featherless AI · Remote (world) · Posted Jan 22, 2026


Transparency

Dead

Online for 43 weeks and never reposted. Jobs this old or reposted this often rarely lead to a hire. It may be filled already.

First seen
Oct 10, 2026
Offered for
302 days

Overview

Category
AI & machine learning
Employment type
Permanent
Workplace type
Remote
Country
United States
Seniority
Senior
Schedule
Full time

Key skills

Inference Performance Optimization
Improve latency, throughput, and cost of large‑scale ML models in production.
GPU/CPU Profiling
Profile and remove bottlenecks in kernel, memory, batching, and I/O pipelines.
Quantization & Pruning
Apply fp16/bf16/int8/fp8 quantization and model pruning for faster inference.
Inference Serving Systems
Build and maintain serving stacks such as Triton or custom runtimes.
PyTorch Model Deployment
Deploy and tune deep‑learning models using PyTorch or equivalent frameworks.
Hardware Benchmarking
Benchmark inference across NVIDIA, AMD GPUs and CPUs in cloud environments.

Key phrases

  • model inference performance at scale
  • profiling systems down to the kernel/GPU level
  • throughput, and cost
  • GPU/CPU inference pipelines
  • Quantization (fp16, bf16, int8, fp8)
  • Speculative decoding, batching, and streaming
  • Model pruning
  • productionize new model architectures
  • Triton, custom runtimes, or bespoke stacks
  • Benchmark performance across hardware
  • TensorRT, ONNX Runtime, vLLM

From the posting

About the Role

We’re looking for a Machine Learning Engineer to own and push the limits of model inference performance at scale. You’ll work at the intersection of research and production—turning cutting-edge models into fast, reliable, and cost-efficient systems that serve real users.

This role is ideal for someone who enjoys deep technical work, profiling systems down to the kernel/GPU level, and translating research ideas into production-grade performance gains.

What You’ll Do

  • Optimize inference latency, throughput, and cost for large-scale ML models in production

  • Profile and bottleneck GPU/CPU inference pipelines (memory, kernels, batching, IO)

  • Implement and tune techniques such as:

    • Quantization (fp16, bf16, int8, fp8)

    • KV-cache optimization & reuse

    • Speculative decoding, batching, and streaming

    • Model pruning or architectural simplifications for inference

  • Collaborate with research engineers to productionize new model architectures

  • Build and maintain inference-serving systems (e.g. Triton, custom runtimes, or bespoke stacks)

  • Benchmark performance across hardware (NVIDIA / AMD GPUs, CPUs) and cloud setups

  • Improve system reliability, observability, and cost efficiency under real workloads

What We’re Looking For

  • Strong experience in ML inference optimization or high-performance ML systems

  • Solid understanding of deep learning internals (attention, memory layout, compute graphs)

  • Hands-on experience with PyTorch (or similar) and model deployment

  • Familiarity with GPU performance tuning (CUDA, ROCm, Triton, or kernel-level optimizations)

  • Experience scaling inference for real users (not just research benchmarks)

  • Comfortable working in fast-moving startup environments with ownership and ambiguity

Nice to Have

  • Experience with LLM or long-context model inference

  • Knowledge of inference frameworks (TensorRT, ONNX Runtime, vLLM, Triton)

  • Experience optimizing across different hardware vendors

  • Open-source contributions in ML systems or inference tooling

  • Background in distributed systems or low-latency services

Why Join Us

  • Real ownership over performance-critical systems

  • Direct impact on product reliability and unit economics

  • Close collaboration with research, infra, and product

  • Competitive compensation + meaningful equity at Series A

  • A team that cares about engineering quality, not hype

More jobs at Featherless AI

Founding Sales Engineer

Featherless AI · Last month | Remote (US & Canada), Atlanta, Tampa +23 | 150000–190000 USD / 1 YEAR | Remote · Mid level · Full time

Create CV

Founding Business Development Rep (AI Cloud US/CA)

Featherless AI · 2 months ago | Remote (US & Canada), Atlanta, Los Angeles +26 | Remote · Junior · Full time

Create CV

Founding Account Executive (AI Cloud)

Featherless AI · 2 months ago | Remote (US & Canada), Atlanta, Los Angeles +17 | $180K–$240K / year | Remote · Mid level · Full time

Create CV

Customer Experience & Community Relations

Featherless AI · 2 months ago | Australia | A$85K–A$103K / year | Remote · Junior · Full time

Create CV

Chief of Staff

Featherless AI · 3 months ago | San Francisco Office, Remote (US & Canada) | Hybrid · Executive · Full time

Create CV
See all 17 jobs at Featherless AI