Machine Learning Engineer

As a Machine Learning Engineer at Etched, your primary responsibilities include implementing high-performance software components for our Kayak inference engine using Rust. You'll be translating mathematical operations from transformer models into optimized sequences tailored for our Sohu chip, developing components that enhance our model assembler and execution pipeline, and implementing efficient batching strategies to improve performance. Additionally, collaboration with hardware engineers and contributions to system architecture evolution are key aspects of this role. You'll also play a part in developing benchmarking tools to track and enhance performance.

To thrive as a Machine Learning Engineer at Etched, you should possess strong software engineering skills, particularly in systems programming with experience in Rust. Familiarity with transformer model architectures and inference-serving stacks is essential, as well as strong mathematical skills, especially in linear algebra. A solid understanding of computational graphs and machine learning workloads is also important. Experience optimizing performance bottlenecks and working cross-functionally within software and hardware domains will make you a standout candidate.

As a Machine Learning Engineer at Etched, you'll engage in a variety of innovative projects. You might optimize operational sequences for specific transformer architectures such as Stable Diffusion or implement efficient memory management strategies for the KV cache. Additionally, you'll build infrastructure for continuous batching and explore novel sampling algorithms for reasoning models. Contributing to the Sohu runtime optimization and performance-related aspects will allow you to significantly impact our generative AI capabilities.

Etched offers a robust benefits package to its Machine Learning Engineers. This includes full medical, dental, and vision insurance, where we cover 100% of premium costs for our employees. We also provide a generous housing subsidy of $2,000/month for team members living in proximity to our Cupertino office. Enjoy complimentary daily lunches and dinners at our office, and for those relocating to join us, we offer relocation support to ease the transition. Our company culture and commitment to employee well-being make Etched a great place to work.

At Etched, we are dedicated to nurturing professional growth for our Machine Learning Engineers. Our collaborative environment encourages cross-functional interactions between engineering and research, allowing you to contribute to both areas. We support participation in open-source projects and actively encourage publishing research papers. This holistic approach helps you expand your skill set and stay at the forefront of AI technology advancements. Our team values innovating and experimenting, making it a thriving place for career development.

When answering this question, highlight your practical experience with Rust, particularly in relation to performance-critical applications. Discuss specific projects where you utilized Rust and the outcomes. Emphasize any optimization techniques or system-level programming tasks you completed, and be ready to demonstrate your understanding of Rust’s unique features, such as memory safety and concurrency.

In your response, outline various strategies you’ve employed to optimize machine learning models. This might include techniques like reducing model complexity, employing quantization, or using efficient batching methods. Provide examples of how these strategies led to performance improvements and any tools or libraries you have experience with for model optimization.

Discuss your familiarity with transformer architectures by mentioning specific models you’ve worked with, such as BERT or GPT. Explain how you used these architectures in past projects and any insights you gained about their advantages and limitations. You can refer to your technical expertise in handling tasks like training, fine-tuning, or deploying these models.

When discussing your debugging approach, highlight your systematic method for diagnosing issues. Describe how you use various tools for logging and monitoring, and potentially how you analyze error rates and performance metrics. Provide examples where your debugging efforts led to significant breakthroughs or improvements in model performance.

Tailor your response to demonstrate your understanding of performance bottlenecks, perhaps by referring to common causes like inefficient data handling, slow model inference, or inadequate resource utilization. Share examples of identifying and resolving bottlenecks in previous projects and how those resolutions benefited model efficiency and speed.

Your answer should reflect any direct experience you’ve had with hardware accelerators such as ASICs or FPGAs. Focus on specific projects where you’ve optimized machine learning workflows using these accelerators. Explain how they impacted model performance and efficiency in your work and any challenges you faced while integrating them.

In your response, discuss the importance of clear communication and mutual respect when working with cross-functional teams. Highlight specific experiences where you collaborated successfully with hardware engineers or product managers, focusing on how shared goals facilitate better outcomes. Be sure to illustrate any strategies you use to overcome potential communication barriers.

Discuss scalability in terms of both data and model size. Explain the methodologies you’ve applied to design scalable machine learning systems, such as distributed processing frameworks, model partitioning, or elastic serving infrastructures. Provide specific outcomes or projects where you successfully scaled an ML system, showcasing the impact of your work.

Provide a concise yet detailed account of a project where you successfully improved inference speed. Discuss the techniques you implemented, such as pruning, quantization, or using more efficient algorithms, and mention the resulting performance metrics. This demonstrates your ability to practically apply optimization techniques effectively.

Share your strategies for staying current, such as following relevant journals, attending conferences, participating in online communities, or regularly engaging with popular AI research blogs. Mention any specific resources or thought leaders you follow that contribute to your ongoing education and inspiration in the field of machine learning.