ML Model Serving Engineer

As a Machine Learning Model Serving Engineer at Sesame, your key responsibilities will include optimizing and deploying real-time scalable ML models, particularly large language models (LLMs) and text-to-speech systems. You'll work with tools like Kubernetes and Ray to enhance our model serving infrastructure and manage deployments on Google Cloud Platform (GCP) using powerful NVIDIA H100 GPUs. Collaborating closely with ML engineers, you'll conduct bottleneck analysis and performance tuning to ensure reliable and efficient ML inference.

To qualify for the Machine Learning Model Serving Engineer role at Sesame, candidates should possess either a Bachelor's or Master's degree in Computer Science, Engineering, or a related field. Importantly, deep experience in deploying and managing scalable machine learning models is essential, along with strong knowledge of cloud platforms, preferably Google Cloud Platform (GCP). Applicants should also demonstrate expertise in performance analysis and optimization, focusing on profiling latency, throughput, and memory usage in ML inference.

While experience with PyTorch is strongly preferred for the Machine Learning Model Serving Engineer role at Sesame, it is not strictly required. However, a solid background in leveraging modern model-serving approaches and working with technologies such as Kubernetes, Ray, and Torch will be beneficial. Familiarity with building large-scale infrastructure and implementing CI/CD pipelines for ML model deployment will enhance your candidacy significantly.

At Sesame, our team culture is grounded in respect, collaboration, and empowerment. We are dedicated to fostering an inclusive workplace where all team members feel valued and appreciated. As a Machine Learning Model Serving Engineer, you will find yourself in a supportive environment, collaborating with talented professionals from diverse backgrounds who share a common goal of revolutionizing how computers interact with us.

Sesame offers a comprehensive benefits package for Machine Learning Model Serving Engineers, which includes 401k matching, 100% employer-paid health, vision, and dental benefits. We also provide unlimited PTO and sick time to support work-life balance, along with matching contributions for flexible spending accounts (FSA) for medical expenses. Our commitment is to ensure our team members are taken care of, both personally and professionally.

When optimizing machine learning models for performance, I first conduct a thorough analysis of the current model's throughput and latency. I then identify bottlenecks, whether they are related to algorithm efficiency, data handling, or infrastructure limitations. Finally, I’ll apply techniques like quantization, pruning, or employing faster inference engines, while continuously monitoring performance improvements.

Certainly! My experience with Google Cloud Platform (GCP) includes deploying machine learning models using its AI services and managing infrastructure with tools like Kubernetes. I have utilized GCP's capabilities to scale applications, ensuring optimal performance while taking advantage of their robust ML tools, including TensorFlow and BigQuery for data management.

I have extensive experience with Kubernetes to automate deployment, scaling, and management of containerized applications. In terms of ML model serving, Kubernetes facilitates the orchestration of services that host our models, enabling us to scale instances based on demand and ensure high availability, which is crucial for real-time ML applications.

To investigate bottlenecks in machine learning inference, I use profiling tools to monitor metrics like latency and resource utilization during model execution. By doing so, I can pinpoint exact stages in the model pipeline that slow down the process, allowing me to optimize algorithms and infrastructure accordingly, and refine performance through tuning and adjustments.

Ensuring the reliability of ML models in production involves implementing robust monitoring systems to track model performance over time. I incorporate automated testing and continuous integration practices to identify and resolve issues early. Furthermore, I maintain logs and establish alerts for unexpected model behavior, allowing for quick interventions when necessary.

I have worked on several projects that involve real-time audio processing, specifically in implementing stream processing frameworks that efficiently manage audio data flow. I focus on maintaining low latency to provide instantaneous feedback for applications such as speech recognition or TTS, ensuring a seamless user experience.

In a recent project, I established a CI/CD pipeline for deploying ML models using tools like GitLab CI and Docker. The pipeline included stages for model testing, validation, and automated rollback in case of failures, ensuring that updates were consistently deployed without downtime, and allowing for rapid iteration based on user feedback.

My strategy includes leveraging profiling tools that provide insights into runtime performance metrics during inference. I typically set up benchmarks to measure throughput and latency under various loads, and assess memory utilization to identify potential overhead, allowing for informed decisions on which model architectures or optimizations to pursue.

I have leveraged infrastructure as code (IaC) tools like Terraform to define and manage cloud resources programmatically, which has streamlined the deployment process for ML models. By using IaC, I can ensure environmental consistency, facilitate reproducibility, and easily roll out changes across different deployment stages in a controlled manner.

I stay updated by actively engaging in professional communities, attending conferences, and participating in webinars. Additionally, I subscribe to relevant research journals and follow thought leaders in the machine learning space on platforms like Medium and GitHub to keep abreast of the latest innovations and best practices in model serving technologies.