Applied ML Engineer - Code Analysis

I’m Brian, the Founder and CEO at Replay.  We’ve recently launched a new API and no-code tool Nut centered on making AIs great at debugging.  We’re building on our five years of history and unique low-overhead Chromium recorder to deliver a product with incredible capabilities that will greatly expand the capabilities of AI developers.  We’re a small company but are well funded by Andreessen Horowitz, IA Ventures, Version One, and other investors.  We’re staffing up to best pursue this opportunity.

The Role

AI generated code often needs to be modified to meet the desired goals of an application or may have bugs when applied to various use cases.  As an applied AI lead, you’ll work directly with our CEO/founder to develop the core technology behind our API that identifies and explains problems in recordings of application code executions.  Advance our technology using AI to identify and resolve bugs in code and to modify AI generated code developed for particular app features and use cases.  Play a significant role in developing AI tools that allow software developers to use AI to write better, more efficient and bug free code.

Responsibilities

• Use LLMs in concert with tools based on traditional program analysis techniques like control dependency analysis and dataflow analysis to advance our technology.

• Review reports shared by users and identify patterns where our API falls short.

• Design improvements to the AI and its tools for solving new problems and building out our evals.

• Optimize the reliability of the API and the consistent performance on our evals.

Your Expertise

• You have a track record building new technology using AI based analysis.

• BS and/or Master/PhD degree, preferably in CS, or equivalent experience

• Passionate about enabling great user experiences using AI - particularly for developers

• Passionate about careful design and construction of robust analyses and evals

• Excellent communication and collaboration skills and the ability to work well within engineering organization and cross functions

• Love collaborating via code reviews, architecture discussions, and are motivated to improve their teammates’ productivity

• Strong product focused coding skills

• Experience working with TypeScript and React applications is highly desired

About Us

We are a fully remote company and this position is open to anyone regardless of location, but we are US based and mainly work during US hours.

Engineering Culture

Our culture is organized around several key values.

• Respect: Respect for you and for each other is crucial. We do our best work as engineers when we deeply understand the problems we’re working on, have ownership over the pieces of the system we’re building, and have the freedom and flexibility to improve them as needed.

• Efficiency: As a small company we follow best practices around CI/CD and have a well organized architecture to ensure we can move quickly. A key part of efficient development is a close relationship with our users, to make sure we’re staying on the right track.

• Supportiveness: Being fully remote it’s important to avoid siloing and make sure we’re working closely together and supporting each other. Work days are organized around a team-wide daily sync and then breakout discussions (live or on discord) to help each other with any issues we’re running into.

• Clarity: A clear vision, direction, and well organized plan are essential for making progress. While our plan will evolve over time as we learn more from our users and partners, at all times we communicate what we’re learning and why any changes are being made.

Interview Process

Our team and interview process want to help you show your best self.  We’ll dive into past projects and simulate working together via pair programming, writing tech specs collaboratively, and talking through decisions (no leetcode or whiteboard exercises). Our interview process is two rounds:

• 30 minute introduction call (we’ll demo Nut and discuss the company’s plans)

• A virtual 2 hour on-site

I’ll be your main point of contact.  You’ll meet me and two of our engineers and have time at each stage to ask questions.

Benefits

• Competitive salary and equity

• 20 days PTO

• If you’re in the US, top notch health insurance for medical, dental, and vision.

• Generous equipment, software, and office furniture budget.

Replay provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, genetics, sexual orientation, gender identity, or gender expression. We are committed to a diverse and inclusive workforce and welcome people from all backgrounds, experiences, perspectives, and abilities.