Staff Software Engineer, Speculative Decoding

About Jobgether

Jobgether is a Talent Matching Platform that partners with companies worldwide to efficiently connect top talent with the right opportunities through AI-driven job matching.

One of our companies is currently looking for a Staff Software Engineer, Speculative Decoding in California.

We’re looking for a seasoned engineer with deep experience in Generative AI inference and a strong command of speculative decoding techniques. In this role, you'll be responsible for developing high-performance, scalable algorithms that enhance speed and accuracy within production-level AI systems. Working in a multi-data center Kubernetes environment, you’ll help design and integrate state-of-the-art decoding methods while driving performance improvements across the inference stack. If you're passionate about transforming leading-edge AI research into production-ready solutions and mentoring others while doing so, this is the ideal opportunity.

Accountabilities:

• Design and implement speculative decoding algorithms to enhance Generative AI inference performance and efficiency.
• Optimize system architecture and software infrastructure for real-time, large-scale AI model deployment.
• Develop and maintain high-performance codebases in C++ and Rust for production-grade distributed systems.
• Work within a multi-process, Kubernetes-based environment utilizing technologies such as MPI.
• Partner with software, research, and operations teams to improve model evaluation, post-training processes, and system scalability.
• Translate recent advancements in AI and speculative decoding into practical, robust implementations.
• Provide technical leadership and contribute to a culture of innovation, mentorship, and continuous improvement.

• Master’s degree in Computer Science, Electrical Engineering, or equivalent practical experience.
• 5+ years of hands-on experience in generative AI inference, particularly with speculative decoding.
• Expertise in C++ with a proven record of building high-performance, distributed systems.
• Familiarity with PyTorch and performance evaluation methodologies for generative models.
• Deep understanding of AI infrastructure challenges, model architecture, and scalable deployment.
• Proficiency with cloud-native tools, Kubernetes environments, and inter-process communication.
• Strong problem-solving abilities, creativity, and collaboration skills in a fast-paced setting.

• Competitive base salary between $175,900 and $307,800 (based on experience)
• Equity participation
• Comprehensive health and wellness benefits
• Flexible work environment with potential site-based requirements
• Continuous learning and growth opportunities
• Inclusive culture committed to diversity, equity, and belonging
• Opportunity to work at the forefront of AI innovation

Jobgether hiring process disclaimer

This job is posted on behalf of one of our partner companies. If you choose to apply, your application will go through our AI-powered 3-step screening process, where we automatically select the 5 best candidates.

Our AI thoroughly analyzes every line of your CV and LinkedIn profile to assess your fit for the role, evaluating each experience in detail. When needed, our team may also conduct a manual review to ensure only the most relevant candidates are considered.

Our process is fair, unbiased, and based solely on qualifications and relevance to the job. Only the best-matching candidates will be selected for the next round.

If you are among the top 5 candidates, you will be notified within 7 days.
If you do not receive feedback after 7 days, it means you were not selected. However, if you wish, we may consider your profile for other similar opportunities that better match your experience.

Thank you for your interest!

#LI-CL1