Let’s get started
By clicking ‘Next’, I agree to the Terms of Service
and Privacy Policy, and consent to receive emails from Rise
Jobs / Job page
Senior Deep Learning Software Engineer, Inference and Model Optimization image - Rise Careers
Job details

Senior Deep Learning Software Engineer, Inference and Model Optimization

NVIDIA is at the forefront of the generative AI revolution! The Algorithmic Model Optimization Team specifically focuses on optimizing generative AI models such as large language models (LLM) and diffusion models for maximal inference efficiency using techniques ranging from neural architecture search and pruning to sparsity, quantization, and automated deployment strategies. Our work includes conducting applied research to improve model efficiency as well as developing an innovative software platform (TRT Model Optimizer). Our software is used both internally across NVIDIA and externally by research and engineering teams alike developing best-in-class AI models.

We are now looking for a Senior Deep Learning Software Engineer to develop and scale up our automated inference and deployment solution. As part of the team, you will be instrumental in pushing the limits of inference efficiency and large-scale, automated deployment. Your work will touch upon fundamental aspects of a typical machine learning stack including working in high-level frameworks like PyTorch and HuggingFace to developing and improving high-performance kernel implementations in CUDA, TRT-LLM, and Triton. This is an exceptional opportunity for passionate software engineers straddling the boundaries of research and engineering, with a strong background in both machine learning fundamentals and software architecture & engineering.

What you’ll be doing:

  • Train, develop, and deploy state-of-the generative AI models like LLMs and diffusion models using NVIDIA's AI software stack.

  • Leverage and build upon the torch 2.0 ecosystem (TorchDynamo, torch.export, torch.compile, etc...) to analyze and extract standardized model graph representation from arbitrary torch models for our automated deployment solution.

  • Develop high-performance optimization techniques for inference, such as automated model sharding techniques (e.g. tensor parallelism, sequence parallelism), efficient attention kernels with kv-caching, and more.

  • Collaborate with teams across NVIDIA to use performant kernel implementations within our automated deployment solution.

  • Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities.

  • Continuously innovate on the inference performance to ensure NVIDIA's inference software solutions (TRT, TRT-LLM, TRT Model Optimizer) can maintain and increase its leadership in the market.

  • Play a pivotal role in architecting and designing a modular and scalable software platform to provide an excellent user experience with broad model support and optimization techniques to increase adoption.

What we need to see:

  • Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field.

  • 5+ years of relevant work or research experience in Deep Learning.

  • Excellent software design skills, including debugging, performance analysis, and test design.

  • Strong proficiency in Python, PyTorch, and related ML tools (e.g. HuggingFace).

  • Strong algorithms and programming fundamentals.

  • Good written and verbal communication skills and the ability to work independently and collaboratively in a fast-paced environment.

Ways to stand out from the crowd:

  • Contributions to PyTorch, JAX, or other Machine Learning Frameworks.

  • Knowledge of GPU architecture and compilation stack, and capability of understanding and debugging end-to-end performance.

  • Familiarity with NVIDIA's deep learning SDKs such as TensorRT.

  • Prior experience in writing high-performance GPU kernels for machine learning workloads in frameworks such as CUDA, CUTLASS, or Triton.

Increasingly known as “the AI computing company” and widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. Are you creative, motivated, and love a challenge? If so, we want to hear from you! Come, join our model optimization group, where you can help build real-time, cost-effective computing platforms driving our success in this exciting and rapidly-growing field.

The base salary range is 184,000 USD - 356,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

NVIDIA Glassdoor Company Review
4.6 Glassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star icon Glassdoor star icon
NVIDIA DE&I Review
No rating Glassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star icon
CEO of NVIDIA
NVIDIA CEO photo
Jensen Huang
Approve of CEO

Average salary estimate

$270250 / YEARLY (est.)
min
max
$184000K
$356500K

If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.

Similar Jobs
Photo of the Rise User
Posted 12 days ago
Customer-Centric
Mission Driven
Inclusive & Diverse
Rise from Within
Diversity of Opinions
Work/Life Harmony
Growth & Learning
Transparent & Candid
Medical Insurance
Paid Time-Off
Maternity Leave
Mental Health Resources
Equity
Child Care stipend
Paternity Leave
WFH Reimbursements
Flex-Friendly
Dental Insurance
Vision Insurance
Life insurance
Health Savings Account (HSA)
Flexible Spending Account (FSA)
401K Matching
Military leave

Exciting opportunity for a Senior Field Applications Engineer at NVIDIA to drive technical engagement and innovation in AI computing systems.

Photo of the Rise User
Posted 12 days ago
Customer-Centric
Mission Driven
Inclusive & Diverse
Rise from Within
Diversity of Opinions
Work/Life Harmony
Growth & Learning
Transparent & Candid
Medical Insurance
Paid Time-Off
Maternity Leave
Mental Health Resources
Equity
Child Care stipend
Paternity Leave
WFH Reimbursements
Flex-Friendly
Dental Insurance
Vision Insurance
Life insurance
Health Savings Account (HSA)
Flexible Spending Account (FSA)
401K Matching
Military leave

Lead NVIDIA's Education Services operations to optimize training systems and workflows using AI-driven strategies in a high-impact leadership role.

Photo of the Rise User
Posted 11 days ago

An innovative AI marketing startup seeks a Senior Backend Engineer with strong Python skills to architect scalable backend systems and mentor a growing engineering team.

Photo of the Rise User

Senior Staff Engineer at Palo Alto Networks, shaping next-generation network security through innovative software solutions in a collaborative hybrid environment.

Photo of the Rise User
Posted 12 days ago

Experienced full stack developer needed to support Boeing's C-17 reliability analytics tools in Warner Robins, GA.

Photo of the Rise User
Arc Hybrid No location specified
Posted 11 days ago

Innovate startup financing at Arc by developing fullstack software features that manage millions in daily transactions within a hybrid NYC work environment.

Photo of the Rise User
Agile Defense Hybrid Colorado Springs, CO
Posted 8 days ago

Agile Defense is hiring a Software Engineer in Colorado Springs to design and implement critical integration enhancements for national security systems.

Photo of the Rise User

Seeking a skilled Java Full Stack Developer with expertise in Tapestry and web services for contract consulting opportunities.

Photo of the Rise User
BitGo Hybrid New York, United States
Posted 3 days ago

BitGo invites a skilled Senior Software Engineer to join their New York team focusing on scalable API development for innovative crypto product growth.

CHAOS Industries Hybrid Hawthorne, California, United States
Posted 6 days ago

CHAOS Inc. seeks a mid-career Command & Control Software Engineer experienced in Python and C2 systems to develop and support cutting-edge mission systems on-site in Hawthorne, CA.

Photo of the Rise User
Zscaler Hybrid San Jose, California, United States
Posted 5 days ago

Experienced Staff Software Engineer - UI Automation needed at Zscaler to lead automation efforts and improve user experience in a hybrid work setting.

Fern Hybrid No location specified
Posted 11 days ago

Fern is looking for a Software Engineer skilled in Go and TypeScript to build and innovate their open source Go SDKs in a fast-paced startup environment.

Photo of the Rise User
Thaloz Hybrid No location specified
Posted 10 days ago

Experienced Frontend Developer wanted to drive the development of an innovative AI-powered mobile app at a San Francisco startup in a part-time capacity.

Photo of the Rise User
Posted yesterday

Lead Salesforce architecture and development at a dynamic global digital engineering company in New York City.

Photo of the Rise User
Mission Driven
Collaboration over Competition
Inclusive & Diverse
Growth & Learning
Maternity Leave
Paternity Leave
Medical Insurance
Dental Insurance
Vision Insurance
Mental Health Resources
Life insurance
Disability Insurance
Health Savings Account (HSA)
Flexible Spending Account (FSA)
401K Matching
Paid Time-Off

Experienced Staff Software Engineer wanted to develop and operate scalable cloud infrastructure and Kubernetes-based compute platforms at Airbnb.

NVIDIA is a publicly traded, multinational technology company headquartered in Santa Clara, California. NVIDIA's invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, and ignited the era of modern AI.

485 jobs
MATCH
VIEW MATCH
BADGES
Badge ChangemakerBadge Diversity ChampionBadge Family FriendlyBadge Global CitizenBadge Work&Life Balance
CULTURE VALUES
Customer-Centric
Mission Driven
Inclusive & Diverse
Rise from Within
Diversity of Opinions
Work/Life Harmony
Growth & Learning
Transparent & Candid
BENEFITS & PERKS
Medical Insurance
Paid Time-Off
Maternity Leave
Mental Health Resources
Equity
Child Care stipend
Paternity Leave
WFH Reimbursements
Flex-Friendly
Dental Insurance
Vision Insurance
Life insurance
Health Savings Account (HSA)
Flexible Spending Account (FSA)
401K Matching
Military leave
FUNDING
SENIORITY LEVEL REQUIREMENT
TEAM SIZE
EMPLOYMENT TYPE
Full-time, onsite
DATE POSTED
May 23, 2025

Subscribe to Rise newsletter

Risa star 🔮 Hi, I'm Risa! Your AI
Career Copilot
Want to see a list of jobs tailored to
you, just ask me below!
LATEST ACTIVITY