Let’s get started
By clicking ‘Next’, I agree to the Terms of Service
and Privacy Policy
Jobs / Job page
Engineering Manager - Inference image - Rise Careers
Job details

Engineering Manager - Inference

In 2012, Lambda started with a crew of AI engineers publishing research at top machine-learning conferences. We began as an AI company built by AI engineers. That hasn't changed. Today, we're on a mission to be the world's top AI computing platform. We equip engineers with the tools to deploy AI that is fast, secure, affordable, and built to scale. Whether they need powerhouse GPU hardware on-site or the flexibility of cloud-based solutions, we've got the horsepower to make it happen. Lambda’s AI Cloud has been adopted by the world’s leading companies and research institutions including Anyscale, Rakuten, The AI Institute, and multiple enterprises with over a trillion dollars of market capitalization. Our goal is to make computation as effortless and ubiquitous as electricity.


If you'd like to build the world's best deep learning cloud, join us. 


*Note: This position requires presence in our San Francisco office location 4 days per week; Lambda’s designated work from home day is currently Tuesday.

What you’ll do

  • Lead our Inference team, developing our rapidly growing inference-as-a-service product as it tackles new types of models, new serving modes, and an ever increasing customer base, across thousands to tens of thousands of GPUs.

  • Be product-focused in your leadership and execution, always placing the needs of the customer first, with a particular focus on feature velocity, reliability and security.

  • Balance development work for cutting-edge features and models against reducing operational overhead and scaling costs.

  • Proactively engage with other teams who run the lower-level infrastructure, datacenter builds, and frontends and work to develop a holistic product and engineering plan.

  • Hire, grow and retain top-tier engineers, in the fields of both distributed systems engineering and machine learning.

  • Shape a culture of sustainable, empathetic, and high-velocity engineering, with a deep focus on cross-team collaboration, documentation, and data-driven decision-making.

You

  • 6+ years in a  full-time management role at a high-growth technology company 

  • 10+ years of industry experience in software engineering, with a focus on large-scale distributed systems and backend systems.

  • Proven record of leading and building engineering teams that work on mission-critical, high performance systems.

  • Exceptional leadership skills that encompass leading by trust, building empathy with your reports and other teams, and maintaining a sustainable but rapid velocity.

  • customer-facing skills, including pre-sales, general support, and incident management.

  • At least one year of experience running inference for machine learning models

  • Knowledge of the machine learning model lifecycle and experience with each of its steps (including pre-training, fine-tuning, and inference)

  • Demonstrated expertise in managing long-term projects alongside urgent, short-term priorities and incident resolution.

  • Extensive experience collaborating with product, sales, and other engineering teams to build cohesive products with a focus on user experience and reliability.

  • Ability to understand, review and structure Python and Go applications.

Nice to Have

  • Significant experience running machine learning workloads on GPUs at large scale

  • Academic or scientific experience working with machine learning

  • Direct experience working with vLLM, sglang, PyTorch, or other ML libraries

  • Experience managing a remote, distributed team

  • Strong experience in Python and/or Go

  • Significant sales, customer service or support experience

Salary Range Information 

Based on market data and other factors, the annual salary range for this position is $300,000-$400,000. However, a salary higher or lower than this range may be appropriate for a candidate whose qualifications differ meaningfully from those listed in the job description. 

About Lambda

  • Founded in 2012, ~350 employees (2024) and growing fast

  • We offer generous cash & equity compensation

  • Our investors include Andra Capital, SGW, Andrej Karpathy, ARK Invest, Fincadia Advisors, G Squared, In-Q-Tel (IQT), KHK & Partners, NVIDIA, Pegatron, Supermicro, Wistron, Wiwynn, US Innovative Technology, Gradient Ventures, Mercato Partners, SVB, 1517, Crescent Cove.

  • We are experiencing extremely high demand for our systems, with quarter over quarter, year over year profitability

  • Our research papers have been accepted into top machine learning and graphics conferences, including NeurIPS, ICCV, SIGGRAPH, and TOG

  • Health, dental, and vision coverage for you and your dependents

  • Commuter/Work from home stipends for select roles

  • 401k Plan with 2% company match (USA employees)

  • Flexible Paid Time Off Plan that we all actually use

A Final Note:

You do not need to match all of the listed expectations to apply for this position. We are committed to building a team with a variety of backgrounds, experiences, and skills.

Equal Opportunity Employer

Lambda is an Equal Opportunity employer. Applicants are considered without regard to race, color, religion, creed, national origin, age, sex, gender, marital status, sexual orientation and identity, genetic information, veteran status, citizenship, or any other factors prohibited by local, state, or federal law.

Lambda Glassdoor Company Review
3.4 Glassdoor star iconGlassdoor star iconGlassdoor star icon Glassdoor star icon Glassdoor star icon
Lambda DE&I Review
No rating Glassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star icon
CEO of Lambda
Lambda CEO photo
Stephen Balaban
Approve of CEO

Average salary estimate

$350000 / YEARLY (est.)
min
max
$300000K
$400000K

If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.

What You Should Know About Engineering Manager - Inference, Lambda

At Lambda, we're on a mission to build the world's best deep learning cloud, and we're looking for an Engineering Manager - Inference to join our passionate team in the heart of San Francisco. Since 2012, we've been pushing the boundaries of AI by developing cutting-edge solutions that empower engineers to deploy models that are fast, secure, and scalable. As an Engineering Manager, you'll lead our Inference team in enhancing our inference-as-a-service product, ensuring it meets the evolving needs of our growing customer base. Your responsibilities will include balancing the agility of feature development with the need for reliability and operational efficiency. You'll work closely with cross-functional teams, driving collaboration and shaping a supportive engineering culture that values high performance and empathy. With at least six years of management experience at a high-growth tech company and considerable expertise in distributed systems engineering, you'll be at the forefront of our mission, helping to hire and nurture top-tier engineering talent. Your technical skills will allow you to manage projects that span both innovative features and critical operational tasks. Join us at Lambda, where you can make a real impact in the AI landscape and help us revolutionize how computation is as commonplace as electricity.

Frequently Asked Questions (FAQs) for Engineering Manager - Inference Role at Lambda
What are the key responsibilities of an Engineering Manager - Inference at Lambda?

As an Engineering Manager - Inference at Lambda, your primary responsibilities will include leading the Inference team to enhance our rapidly growing inference-as-a-service product. You'll ensure that the product meets customer needs, focusing on feature velocity, reliability, and security. Additionally, you'll balance the development of cutting-edge features with operational efficiency and manage collaboration across teams to deliver a holistic engineering plan.

Join Rise to see the full answer
What qualifications are needed for the Engineering Manager - Inference role at Lambda?

To be considered for the Engineering Manager - Inference position at Lambda, you should have at least six years of experience in a management role within a high-growth technology environment and over ten years in software engineering with a specialization in large-scale distributed systems. A solid understanding of the machine learning model lifecycle and experience in managing engineering teams working on high-performance systems is essential.

Join Rise to see the full answer
How can I prepare for an Engineering Manager - Inference interview at Lambda?

Preparing for an interview for the Engineering Manager - Inference role at Lambda involves familiarizing yourself with leadership in tech, particularly in distributed systems and machine learning. It's important to reflect on past experiences where you successfully led teams and tackled complex projects. Understanding Lambda's mission and the specific expectations of the role will also give you an edge in the interview process.

Join Rise to see the full answer
What is the company culture like for an Engineering Manager - Inference at Lambda?

At Lambda, the culture is centered around collaboration, empathy, and high-velocity engineering. As an Engineering Manager - Inference, you'll play a critical role in fostering a culture that supports innovation while valuing documentation and data-driven decision-making. The company promotes a flexible and supportive work environment, encouraging employees to grow and excel.

Join Rise to see the full answer
What growth opportunities exist for an Engineering Manager - Inference at Lambda?

An Engineering Manager - Inference at Lambda will find numerous opportunities for professional growth. With a commitment to sustainable engineering practices and a focus on team development, you'll be encouraged to lead projects that influence the direction of our inference product. You'll also have the chance to collaborate across departments, gaining experience in different areas of the business.

Join Rise to see the full answer
Common Interview Questions for Engineering Manager - Inference
How do you prioritize tasks as an Engineering Manager - Inference?

In this role, prioritizing tasks involves assessing both long-term project goals and immediate needs. Communicate closely with your team to understand their workloads and challenges, and leverage data to make informed decisions on where your focus should be, always aligning with customer needs and business objectives.

Join Rise to see the full answer
Can you describe your experience with managing machine learning inference workflows?

When discussing your experience with machine learning inference workflows, highlight specific projects where you guided your team through the intricacies of model lifecycle management, including pre-training, fine-tuning, and ultimately delivering predictions efficiently. Be sure to discuss techniques you implemented to optimize these workflows.

Join Rise to see the full answer
What strategies do you use to ensure effective communication within your team?

Effective communication strategies may include regular check-ins, utilizing collaborative tools for project updates, and fostering an open-door policy. Encourage feedback and promote an atmosphere where team members can share their insights and concerns without hesitation to enhance collaboration.

Join Rise to see the full answer
How do you approach hiring top-tier engineers for your team?

When hiring for your team, focus on creating a thorough and structured interview process that emphasizes both technical proficiency and cultural fit. Leverage your network to seek recommendations, and ensure you communicate Lambda's mission and values to attract candidates who are truly passionate about AI.

Join Rise to see the full answer
How do you handle conflicts between product requirements and engineering limitations?

Handling conflicts involves fostering transparent discussions between product and engineering teams to align expectations. Ensure that you listen actively to both sides, and explore creative solutions that can satisfy product goals while respecting engineering constraints.

Join Rise to see the full answer
What role does documentation play in your engineering management style?

Documentation is crucial in engineering management, as it preserves knowledge and ensures consistency across projects. Encourage your team to regularly update documentation for both processes and decisions, making it easier to onboard new team members and providing clarity for ongoing work.

Join Rise to see the full answer
How do you stay current with technology trends in AI and machine learning?

To stay updated on technology trends, regularly read industry publications, attend relevant conferences, participate in webinars, and engage with the AI community through forums and social media. Sharing insights with your team can foster a culture of learning and innovation.

Join Rise to see the full answer
What challenges have you faced in previous engineering management roles?

Discuss specific challenges such as team dynamics, project delivery under tight deadlines, or managing remote teams. Highlight how you approached these challenges with proactivity and how you leveraged them as opportunities for growth and improvement.

Join Rise to see the full answer
How do you ensure your team maintains a sustainable work pace?

Sustainability in work pace is achieved by monitoring workloads, encouraging breaks, and promoting a healthy work-life balance. Regular one-on-ones help you gauge employee well-being and make adjustments to project timelines as needed to prevent burnout.

Join Rise to see the full answer
Why is empathy an important quality for an Engineering Manager - Inference?

Empathy enables an Engineering Manager to connect with team members and understand their perspectives. It's essential for building trust, fostering a collaborative environment, and leading effectively, especially in high-pressure situations where team morale can impact performance.

Join Rise to see the full answer
Similar Jobs
Photo of the Rise User

Join Lambda as a Senior Site Reliability Engineer to architect highly reliable and efficient cloud-native solutions for advanced AI computing.

Photo of the Rise User
Posted 6 days ago

Lambda is looking for a seasoned Principal in Enterprise Business Development to leverage your extensive experience in AI and technology sales to strengthen relationships with major enterprises.

Photo of the Rise User
Customer-Centric
Mission Driven
Inclusive & Diverse
Rise from Within
Diversity of Opinions
Work/Life Harmony
Growth & Learning
Transparent & Candid
Medical Insurance
Paid Time-Off
Maternity Leave
Mental Health Resources
Equity
Child Care stipend
Paternity Leave
WFH Reimbursements
Flex-Friendly
Dental Insurance
Vision Insurance
Life insurance
Health Savings Account (HSA)
Flexible Spending Account (FSA)
401K Matching
Military leave

NVIDIA invites innovative new graduates to join their team as a Circuit Design Engineer, focusing on SRAM design and automation.

Photo of the Rise User
fairlife Hybrid Webster, New York, United States
Posted 5 days ago

Join fairlife, LLC as a Packaging Project Manager to drive innovative packaging solutions and ensure operational efficiency.

Photo of the Rise User

As an Engineering Manager at Chainlink Labs, you'll lead a talented team focused on developing robust deployment tools for cutting-edge blockchain applications.

Photo of the Rise User
SGS Hybrid 775 Montague Expy, Milpitas, CA 95035, USA
Posted 3 days ago

Become a vital part of SGS, where your expertise as a Senior Engineer will enhance safety evaluations and inspections globally.

Come elevate your career as a Bridge Engineering Principal with CEC, where innovation meets a supportive team atmosphere.

Posted 2 days ago

AtkinsRéalis is hiring a Marine Engineering Director to spearhead innovative engineering projects in Saudi Arabia's NEOM initiative.

Photo of the Rise User
NBCUniversal Remote 1000 Flower St, Glendale, CALIFORNIA
Posted 12 days ago

Be a part of DreamWorks Animation as a Senior Platform Engineer, contributing to cutting-edge platform solutions that support creative storytelling.

Photo of the Rise User
Factor Remote Poland - Remote
Posted 11 days ago

As an AI Engineer at Factor, you'll spearhead the development of generative AI solutions in a dynamic R&D environment.

Lambda provides Artificial Intelligence and Machine Learning infrastructure to companies like Apple, Intel, Microsoft, MIT, Harvard, the Federal Government, and the DOD. Were headquartered in the Dogpatch and are a short walk from the 22nd Street ...

70 jobs
MATCH
VIEW MATCH
FUNDING
DEPARTMENTS
SENIORITY LEVEL REQUIREMENT
TEAM SIZE
EMPLOYMENT TYPE
Full-time, hybrid
DATE POSTED
April 9, 2025

Subscribe to Rise newsletter

Risa star 🔮 Hi, I'm Risa! Your AI
Career Copilot
Want to see a list of jobs tailored to
you, just ask me below!
LATEST ACTIVITY
Photo of the Rise User
Someone from OH, Pickerington just viewed Senior Business Analyst (Salesforce) at Protolabs
H
Someone from OH, Akron just viewed Brand Marketing Manager at Huntington
R
Someone from OH, Hamilton just viewed Forklift Operator Warehouse at Ryder
Photo of the Rise User
Someone from OH, Cincinnati just viewed Ad Ops Specialist, Display at System1
Photo of the Rise User
Someone from OH, Cincinnati just viewed FQHC Billing & Collections Manager at OhioGuidestone
Photo of the Rise User
Someone from OH, Cleveland just viewed Enrollment Specialist- Remote at Adtalem Global Education
o
Someone from OH, Dayton just viewed Marketing and Communications Specialist at osu
Photo of the Rise User
Someone from OH, Columbus just viewed Construction Coordinator at Meijer
Photo of the Rise User
Someone from OH, Steubenville just viewed Legal & Compliance Internship at Smiths Group
Photo of the Rise User
Someone from OH, Warren just viewed Senior Front-End Developer at Worldly
Photo of the Rise User
Someone from OH, Tiffin just viewed Game Operations Specialist at Genius Sports
u
Someone from OH, Loveland just viewed Customer Service Agent - Part Time at uhaul
Photo of the Rise User
Someone from OH, Cleveland just viewed HR Manager at Shearer's Foods
Photo of the Rise User
Someone from OH, Columbus just viewed Mid Level, System Administrator - (ETS) at Delivery Hero
Photo of the Rise User
Someone from OH, Mason just viewed Inside Sales Co-Op at VEGA Americas
Photo of the Rise User
Someone from OH, Sandusky just viewed Director of IT at Kyo
Photo of the Rise User
Someone from OH, Delaware just viewed Practice Group Manager at LifeStance Health
Photo of the Rise User
6 people applied to Machinist Apprentice at LLNL