Infrastructure Engineer (GPU Cluster)

As an Infrastructure Engineer (GPU Cluster) at Pallon, you will have diverse responsibilities that include designing and building custom GPU clusters for deep learning, managing both on-prem and cloud infrastructure, troubleshooting system issues, and ensuring the smooth operation of all infrastructure components. You'll play a pivotal role in optimizing performance and scaling our systems to meet real-world demands.

To qualify for the Infrastructure Engineer (GPU Cluster) role at Pallon, candidates should have over 5 years of experience managing infrastructure in dynamic environments, especially startups. A solid understanding of Linux, networking, and distributed systems is essential. Familiarity with coding to automate and debug tasks is important, alongside a university degree in Computer Science or a relevant field.

Success as an Infrastructure Engineer (GPU Cluster) at Pallon hinges on a combination of technical skills and personal attributes. Strong Linux fundamentals, coding ability, and knowledge of cloud-native tools are vital. Equally important are excellent communication skills and the ability to collaborate well with cross-disciplinary teams, especially engineers focused on areas outside of infrastructure.

The work environment at Pallon is collaborative and innovative, promoting both autonomy and cooperation among team members. You'll be part of a motivated, intelligent, and supportive team, focused on making a real difference in urban sustainability through technology. Flexibility is also valued, with options for remote work and access to our office in Zürich.

At Pallon, the technology stack for the Infrastructure Engineer (GPU Cluster) position includes a high-performance compute cluster set up with Linux, NVidia GPUs, and Slurm. For cloud services, we utilize Google Cloud Platform, employing Kubernetes and Docker, as well as CI/CD through GitLab. Familiarity with this tech stack is beneficial but not mandatory.

To effectively answer this question, reflect on any specific projects you have worked on that involved GPU clusters. Discuss the challenges faced, the solutions you implemented, and any operational metrics that improved as a result. Highlight your understanding of performance optimization and troubleshooting methodologies used in GPU workloads.

Discuss the strategies you employ for managing hybrid infrastructure environments, focusing on how you balance scalability, cost, and performance. Make sure to mention any tools or methodologies used in your past roles, including your experience with automation and monitoring solutions like Terraform, Ansible, or custom scripts.

Share a specific incident, outlining how you identified the problem, the steps taken to troubleshoot, and the ultimate solution achieved. Emphasize your analytical thinking and methodical approach, showcasing any teamwork involved in navigating the challenge effectively.

Mention your knowledge of performance metrics and what tools or methodologies you've used for optimization, such as effective GPU scheduling, adjusting batch sizes, and network tuning. Talk about how you balance resource allocation between various workloads to ensure continuous performance improvements.

Identify the programming languages you have experience with, such as Python, Bash, or Go, and explain how you've used them to automate and streamline your infrastructure tasks. Provide specific examples of scripts or tools you've developed to improve efficiency and reliability in your past roles.

Detail your understanding of best practices for securing infrastructure, including access control, data encryption, and regular system audits. Mention any specific tools or frameworks used to manage security protocols effectively within both on-premises and cloud environments.

Share your hands-on experience with Kubernetes in managing containerized applications and workloads. Highlight specific tasks you have accomplished, such as deploying applications, setting up service meshes, or managing stateful applications, indicating your comfort level with Kubernetes in various scenarios.

Discuss the resources you leverage for continuous learning, such as tech blogs, online courses, webinars, and participation in relevant forums or communities. Share any recent trends in infrastructure that you've explored and how you’ve applied new knowledge to your work.

Describe your approach to task prioritization, emphasizing your ability to evaluate urgency and impact on productivity. Discuss any frameworks you follow, such as Eisenhower Matrix, or tools you use for tracking tasks, ensuring stakeholders are informed about the status of critical infrastructure initiatives regularly.

Provide an example of a cross-functional project where collaboration was key. Emphasize your communication strategies and how you aligned objectives with other teams, such as developers or data scientists, to achieve a common goal while addressing any technical challenges that arose.