Senior Infrastructure Engineer

The Senior Infrastructure Engineer at Pallon is responsible for architecting and managing our complete infrastructure, which includes a high-density compute cluster and a cloud environment. This involves designing and optimizing a powerful GPU cluster for deep learning and computer vision, collaborating with engineering teams to address infrastructure challenges, and ensuring high security and performance of our production systems and data pipelines. A hands-on approach is essential, as you’ll handle everything from troubleshooting issues to strategic planning.

To become a Senior Infrastructure Engineer at Pallon, you should have a university degree in Computer Science or a related field and possess over 5 years of experience in architecting and implementing infrastructure solutions, ideally in startup settings. A deep understanding of Linux system administration, cloud infrastructure, hardware systems, and proficiency in scripting or programming is crucial. Familiarity with high-performance computing environments and data engineering is a bonus.

In the role of Senior Infrastructure Engineer at Pallon, you will work with a diverse tech stack that includes high-performance computing tools such as Linux, Nvidia GPUs, Slurm, and Infiniband. You’ll also engage with cloud technologies like Google Cloud Platform, Kubernetes, Docker, and CI/CD tools such as GitLab. Additionally, experience with data analytics tools like DBT, BigQuery, and Metabase will be beneficial.

The team culture at Pallon is collaborative, supportive, and fun! As a Senior Infrastructure Engineer, you'll join a motivated group of software engineers and AI researchers who are passionate about creating impactful solutions for urban sustainability. The culture encourages innovation, from virtual exploration of sewers to gathering for offsite team-building activities. Plus, you have the flexibility to work remotely or from our beautiful Zürich office.

Pallon values equality of opportunity and inclusivity in its hiring process. We actively encourage women and candidates from under-represented backgrounds to apply for the Senior Infrastructure Engineer position, even if they may not meet 100% of the requirements. The goal is to build a diverse team that fosters different perspectives and ideas, aligning with our mission to create sustainable technological solutions.

When answering this question, highlight specific cloud technologies you’ve managed, like Google Cloud Platform or AWS. Discuss your role in designing and implementing solutions, your understanding of container orchestration like Kubernetes, and how you ensured system reliability and performance. Providing examples of challenges you overcame will also demonstrate your problem-solving skills.

Discuss techniques such as workload distribution, optimizing algorithms, and proper configuration of the cluster components. Mention tools you’ve used for performance monitoring and tuning, and provide specific examples of enhancements made in previous roles to illustrate your hands-on experience with GPU clusters.

You can highlight strategies like setting up load balancers, implementing failover solutions, and regularly conducting performance testing. Explain how you monitor systems for potential issues and engage in proactive maintenance to minimize risks, showcasing your commitment to system reliability.

Pick a specific scenario where you faced a significant challenge, such as troubleshooting a Kubernetes pod eviction issue. Describe the steps you took, the rationale behind your decisions, and the ultimate resolution, illustrating your analytical skills and ability to work under pressure.

Emphasize the importance of communication in your collaboration approach. You should explain how you take the time to understand the needs and challenges faced by engineering teams and how you turn their requirements into effective infrastructure solutions, promoting a positive and productive collaboration environment.

Discuss the resources you utilize to stay updated, such as online courses, webinars, tech blogs, and industry forums. Illustrate your eagerness to learn through examples of new technologies you've recently adopted or explored and how they might benefit your work at Pallon.

Speak about your familiarity with tools like Terraform or Ansible. Explain how you've utilized these tools to automate and manage infrastructure setups and how this practice has improved the efficiency and consistency of deployments in your previous roles.

If applicable, describe your roles in managing or optimizing high-performance computing clusters. Key points can include your familiarity with associated software, challenges faced in these environments, and how your actions contributed to successful outcomes.

Describe the tools you use for performance monitoring, such as Prometheus or Grafana. Discuss how you analyze performance metrics to preemptively identify issues and how you respond to alerts and performance degradation rapidly, ensuring minimal disruption.

Convey your enthusiasm for Pallon’s mission to create sustainable urban environments through innovative technology. Discuss how your values align with the company's goals and your excitement about contributing to a positive impact on society and the environment through your skills in infrastructure engineering.