Member of Technical Staff - Training Infrastructure Engineer
Liquid AI, an MIT spin-off, is a foundation model company headquartered in Boston, Massachusetts. Our mission is to build capable and efficient general-purpose AI systems at every scale.
Our goal at Liquid is to build the most capable AI systems to solve problems at every scale, such that users can build, access, and control their AI solutions. This is to ensure that AI will get meaningfully, reliably and efficiently integrated at all enterprises. Long term, Liquid will create and deploy frontier-AI-powered solutions that are available to everyone.
What This Role Is
We're looking for a Training Infrastructure Engineer to design, build, and optimize the distributed systems that power our Liquid Foundation Models (LFMs). This is a highly technical role focused on creating the scalable infrastructure that enables efficient training of models across the spectrum—from compact specialized models to massive multimodal systems—while maximizing hardware utilization and minimizing training time.
- You have extensive experience building distributed training infrastructure for language and multimodal models, with hands-on expertise in frameworks like PyTorch Distributed, DeepSpeed, or Megatron-LM
- You're passionate about solving complex systems challenges in large-scale model training—from efficient multimodal data loading to sophisticated sharding strategies to robust checkpointing mechanisms
- You have a deep understanding of hardware accelerators and networking topologies, with the ability to optimize communication patterns for different parallelism strategies
- You're skilled at identifying and resolving performance bottlenecks in training pipelines, whether they occur in data loading, computation, or communication between nodes
- You have experience working with diverse data types (text, images, video, audio) and can build data pipelines that handle heterogeneous inputs efficiently
- You've implemented custom sharding techniques (tensor/pipeline/data parallelism) to scale training across distributed GPU clusters of varying sizes
- You have experience optimizing data pipelines for multimodal datasets with sophisticated preprocessing requirements
- You've built fault-tolerant checkpointing systems that can handle complex model states while minimizing training interruptions
- You've contributed to open-source training infrastructure projects or frameworks
- You've designed training infrastructure that works efficiently for both parameter-efficient specialized models and massive multimodal systems
- Design and implement high-performance, scalable training infrastructure that efficiently utilizes our GPU clusters for both specialized and large-scale multimodal models
- Build robust data loading systems that eliminate I/O bottlenecks and enable training on diverse multimodal datasets
- Develop sophisticated checkpointing mechanisms that balance memory constraints with recovery needs across different model scales
- Optimize communication patterns between nodes to minimize the overhead of distributed training for long-running experiments
- Collaborate with ML engineers to implement new model architectures and training algorithms at scale
- Create monitoring and debugging tools to ensure training stability and resource efficiency across our infrastructure
- The opportunity to solve some of the hardest systems challenges in AI, working at the intersection of distributed systems and cutting-edge multimodal machine learning
- Experience building infrastructure that powers the next generation of foundation models across the full spectrum of model scales
- The satisfaction of seeing your work directly enable breakthroughs in model capabilities and performance
Average salary estimate
If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.
At Liquid AI, an innovative MIT spin-off based in Boston, we are on a mission to revolutionize AI technology, and we’re looking for a talented Member of Technical Staff - Training Infrastructure Engineer to join our team. In this pivotal role, you will design, build, and optimize the intricate distributed systems that power our groundbreaking Liquid Foundation Models (LFMs). Your talents will be crucial as you tackle the exciting challenges of creating scalable infrastructure that facilitates efficient training of various models—from compact specialized ones to expansive multimodal systems. If you're passionate about maximizing hardware utilization and minimizing training times while embracing the complexities of systems challenges, this could be the perfect fit for you. You will collaborate with other ML engineers, diving deep into efficient multimodal data loading, sharding strategies, and robust checkpointing mechanisms. With your expertise in frameworks like PyTorch Distributed and DeepSpeed, you’ll enhance our training infrastructure and ensure our AI solutions are powerfully integrated into enterprises worldwide. By joining Liquid AI, you’ll not only work on cutting-edge technology but also gain invaluable experience that will shape the future of AI development and deployment. Let's build the future together!
Subscribe to Rise newsletter
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
Sign up for our weekly
newsletter of fresh jobs
