Software Engineer, Inference Infrastructure

Rhoda AI · Mountain View · Posted May 12, 2026


Transparency

Dead

Online for 27 weeks and never reposted. Jobs this old or reposted this often rarely lead to a hire. It may be filled already.

First seen
Oct 10, 2026
Offered for
192 days

Overview

Category
AI & machine learning
Employment type
Permanent
Workplace type
On-site
Country
United States
Seniority
Mid level
Schedule
Full time
Salary
$175,000–$280,000 / year, plus equity and cash value

From the posting

At Rhoda AI, we’re building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots. Our robots are designed to be generalists capable of operating in complex, real-world environments and handling long-tail edge cases, made possible by our cutting edge research and end-to-end system design. We've raised over $450M and are investing aggressively in model research, infrastructure, hardware development, and manufacturing scale-up to make generalist robotics a reality.

We're looking for a Software Engineer to help build and operate the systems that power our model deployment stack. You'll be responsible for running large foundation models efficiently and reliably across cloud and on-prem environments, with a focus on resource management, scheduling, and infrastructure scalability.

What You'll Do

  • Design and operate large-scale infrastructure to run model workloads across cloud and on-prem environments

  • Build and maintain Kubernetes-based deployment pipelines for managing distributed ML workloads

  • Own resource scheduling and orchestration across GPU clusters — optimizing utilization, workload balancing, and cost-performance tradeoffs

  • Integrate and manage ML frameworks and model serving systems (e.g., Triton, Ray Serve, TorchServe) across research and production use cases

  • Build tooling for model deployment, versioning, and observability to support fast iteration cycles

  • Contribute to the reliability and scalability of the infrastructure stack as model complexity and deployment footprint grow

What We're Looking For

  • 3+ years of experience in ML infrastructure, MLOps, or distributed systems

  • Strong proficiency with Kubernetes and containerized deployment pipelines

  • Experience with GPU orchestration and resource scheduling across large distributed jobs

  • Experience with cloud providers (e.g., AWS, GCP) and hybrid cloud/on-prem infrastructure

  • Familiarity with ML frameworks (e.g., PyTorch, JAX) and model serving tools (e.g., Triton, Ray Serve, TorchServe)

  • Strong debugging instincts and ownership mentality — comfortable driving issues to resolution across the stack

Nice to Have (But Not Required)

  • Experience with streaming systems or high-throughput data transport (e.g., Kafka, gRPC, NATS)

  • Background in networking, low-latency systems, or network-aware scheduling

  • Experience with edge/cloud hybrid deployment patterns and the latency constraints that come with them

  • Familiarity with on-robot or embedded inference environments

  • Experience with large-scale cluster topology and scheduling systems (e.g., SLURM, Ray, Volcano)

Why This Role

  • Own the infrastructure layer that connects our foundation models to real robot behavior — a direct line between your work and what the robot does in the world

  • Be part of building the infrastructure stack for one of the most technically ambitious robotics companies in the world

More jobs at Rhoda AI

Technical Talent Partner

Rhoda AI · 2 days ago | Sunnyvale, CA | On-site · Senior · Full time

Create CV

Research Intern

Rhoda AI · Last week | Mountain View | On-site · Intern · Full time

Create CV

Software QA Engineer

Rhoda AI · Last week | Mountain View | On-site · Mid level · Full time

Create CV

Optical Sensing Intern - Robot Perception

Rhoda AI · Last week | Mountain View | On-site · Intern · Full time

Create CV

Senior/Staff TPM - Hardware

Rhoda AI · 2 weeks ago | Mountain View | EquityPercentage; 175000–280000 USD / 1 YEAR | On-site · Senior · Full time

Create CV
See all 40 jobs at Rhoda AI