Member of Technical Staff - ML Inference Engineer

We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Join us in shaping the future of AI and pushing the boundaries of what's possible in video generation.

Role overview:

We are looking for a senior software engineer to join our inference team. In this role, you will be responsible for designing and scaling our inference systems as they grow to support over 1 million users across more than 20 different data centers.

Key responsibilities:

• Develop high-performance, high-throughput, efficient, and low-latency inference pipelines.

• Design, develop, and maintain scalable backend services that support our AI-powered content creation platform.

• Implement and optimize model serving infrastructure using Kubernetes and other cloud-native technologies.

• Collaborate with ML engineers to transition models from research to production.

• Design APIs for integrating our AI capabilities into our partner ecosystem.

• Implement monitoring, logging, and alerting systems for backend services and model inference.

• Develop monitoring infrastructure for our ML serving pipeline and apply advanced model compression and optimization techniques (quantization, pruning, distillation) to improve inference performance.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field

• 5+ years of experience in software engineering, with at least 3 years focusing on backend systems and ML infrastructure

• Strong past experience with Ray or Kubernetes

• Strong proficiency in Python and Go

• Experience with GPU programming is a plus

• Solid understanding of model serving frameworks (e.g., TensorFlow Serving, NVIDIA Triton)

• Experience with a ML framework such as TensorFlow, PyTorch, or JAX

• Experience with model compression and optimization techniques

• Strong knowledge of cloud platforms (AWS, GCP, or Azure) and their ML-specific services

• Familiarity with distributed systems and microservices architectures

• Experience with high-performance, low-latency systems

Genmo is an Equal Opportunity Employer. Candidates are evaluated without regard to age, race, color, religion, sex, disability, national origin, sexual orientation, veteran status, or any other characteristic protected by federal or state law. Genmo, Inc. is an E-Verify company and you may review the Notice of E-Verify Participation and the Right to Work posters in English and Spanish.