Software Manager, AI Infrastructure System

NVIDIA has continuously reinvented itself over two decades. Our invention of the GPU in 1999 fueled the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI and enabled the next era of computing. NVIDIA is a “learning machine” that constantly evolves by adapting to new opportunities that are hard to address, that matters to the world, and that only we can address. This is our life’s work, to amplify human imagination and intelligence, and expand what is possible. We’re seeking strategic, bold, hard-working, and creative individuals who are passionate about helping us tackle challenges no one else can solve. Make the choice to join us today. 

We are looking for an AI Infrastructure System Software Manager to join our mission to continue improving our HPC infrastructure. Our team builds and operates sophisticated infrastructure to enable business critical services and AI applications. You will be working with a team of passionate and skilled engineers that are continuously working to provide better tools to build and manage this infrastructure. Ideal candidate is strong in software development, designing and creating reliable distributed systems, and has the ability to implement well thought out long term maintenance strategy. 

What you'll be doing: 

• Mentor, grow, and develop a world-class team of AI infrastructure engineers. 

• Work across several teams and orgs to build products that use LLMs and agent systems to serve the needs of NVIDIA engineering teams. In that role, you will be collaborating with research and infra teams and serve a large user base (hardware/software teams across NVIDIA).

• Align priorities across collaborators and define metrics for measuring the success of the product/team.

• Develop and execute strategies for scalable, reliable, and secure AI infrastructure supporting both research and production workloads.  

• Ensure robust monitoring, logging, visualization, and alerting capabilities to guarantee promised uptime and operational excellence.

• Architect, design, develop, and maintain infrastructure and large-scale applications for LLM-based solutions. Optimize these systems for performance, scalability, reliability, and secure data management.

• Stay updated with the latest trends in AI, ML, and infrastructure, proactively seeking opportunities to integrate advancements into Nvidia’s LLM and AI infrastructure solutions.

What we need to see: 

• 10+ overall years of industry large distributed system software development experience.

• BS+ degree in CS or related/equivalent experience. 

• 5+ years of experience managing of AI and SW development teams.

• Familiarity with modern software development stacks and tools, including containerization, cloud or on-premises deployments, API integration for seamless model operation, and real-time processing frameworks.   

• Experience in developing and maintaining LLM or GenAI infrastructure  

• Excellent communication, collaboration and problem-solving skills, with a dedication to encouraging an inclusive and diverse workplace.  

• Hands-on experience developing large-scale distributed systems 

Ways to stand out from the crowd: 

• Strong technical background in cloud/distributed infrastructure 

• Experience debugging functional and performance issues in HPC GPU clusters 

• Background in running and instrumenting distributed LLM training on a multi GPU HPC cluster  

• Experience with HPC schedulers such as Slurm 

Widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/