Senior QA Engineer - AI & LLM

Matillion is The Data Productivity Cloud.

We are on a mission to power the data productivity of our customers and the world, by helping teams get data business ready, faster. Our technology allows customers to load, transform, sync and orchestrate their data. 

We are looking for passionate, high-integrity individuals to help us scale up our growing business. Together, we can make a dent in the universe bigger than ourselves.

With offices in the UK, US and Spain, we are now thrilled to announce the opening of our new office in Hyderabad, India. This marks an exciting milestone in our global expansion, and we are now looking for talented professionals to join us as part of our founding team.

About the Role

We’re on the hunt for a Senior Quality Engineer to help us push the boundaries of SaaS quality assurance while integrating cutting-edge Generative AI (GenAI) technologies. At the heart of our engineering culture is a commitment to delivering high-quality, scalable, and secure software – and we believe quality should be a shared responsibility across the team.

As a Senior QA Engineer, you'll play a pivotal role in shaping and maintaining quality throughout the SDLC – combining deep QA expertise with a passion for emerging AI technologies. You’ll work hands-on with cross-functional teams, take ownership of our testing strategies, and guide the adoption of best practices across automation, GenAI evaluations, and testing frameworks.

• Champion a shift-left testing culture by embedding quality across every phase of our software development lifecycle.
• Design, build, and maintain robust test automation frameworks for functional, integration, performance, security, and accessibility testing.
• Collaborate closely with product managers, developers, and AI engineers to test and evaluate LLM-integrated features using both traditional and GenAI-specific testing methods.
• Implement tools and techniques to test LLM-based functionality, including prompt evaluation, RAG-enhanced responses, and API behavior.
• Drive the use of modern CI/CD pipelines, ensuring testing is efficient, scalable, and automated wherever possible.
• Analyze, interpret, and share meaningful insights from test results – especially when comparing LLM model responses, prompt outcomes, and system behavior.
• Mentor team members in test strategies, automation practices, and GenAI testing methodologies.
• Continuously assess and improve the quality landscape, identifying opportunities to increase speed, accuracy, and innovation in how we test.

• Core QA Expertise
• Strong experience with end-to-end testing of distributed systems and SaaS applications.
• Deep understanding of testing methodologies including exploratory, regression, risk-based, performance, and security testing.
• Skilled in writing automated tests using tools like Cypress, Pact, or similar frameworks, with coding experience in Java/JavaScript.
• Solid understanding of the Test Pyramid, CI/CD pipelines, and agile methodologies (Scrum, Kanban).
• Experience working with cloud platforms like AWS, GCP, or Azure, plus comfort with SQL and database testing.
• A team player with a QA-first mindset and a drive to elevate engineering standards across the board.

• GenAI/LLM Knowledge (Essential)
• Hands-on experience working with LLM APIs like OpenAI, Anthropic, or HuggingFace.
• Familiarity with prompt engineering, fine-tuning prompts for desired outcomes.
• Practical experience using Retrieval-Augmented Generation (RAG) to enrich LLM responses.
• Understanding of key GenAI concepts like system/user prompts, tokens, embeddings, temperature, top-p, context windows, and stop sequences.

• Nice-to-Have
• Experience with fine-tuning LLMs or training smaller models for specific tasks.
• Familiarity with prompt compression techniques and advanced GenAI patterns like prompt chaining, agentic workflows, or LLM routing.
• Exposure to abstraction libraries such as Spring AI or similar.
• Experience with LLM eval frameworks to benchmark model performance and prompt effectiveness.

Matillion has fostered a culture that is collaborative, fast-paced, ambitious, and transparent, and an environment where people genuinely care about their colleagues and communities. 

Our 6 core values guide how we work together and with our customers and partners. We operate a truly flexible and hybrid working culture that promotes work-life balance, and are proud to be able to offer the following benefits:

- Company Equity

- 27 days paid time off

- 12 days of Company Holiday

- 5 days paid volunteering leave

- Group Mediclaim (GMC)

- Enhanced parental leave policies

- MacBook Pro

- Access to various tools to aid your career development

More about Matillion

Thousands of enterprises including Cisco, DocuSign, Slack, and TUI trust Matillion technology to load, transform, sync, and orchestrate their data for a wide range of use cases from insights and operational analytics, to data science, machine learning, and AI. 

With over $300M raised from top Silicon Valley investors, we are on a mission to power the data productivity of our customers and the world.

We are passionate about doing things in a smart, considerate way. We’re honoured to be named a great place to work for several years running by multiple industry research firms. 

We are dual headquartered in Manchester, UK and Denver, Colorado.

We are keen to hear from prospective Matillioners, so even if you don’t feel you match all the criteria please apply and a member of our Talent Acquisition team will be in touch. Alternatively, if you are interested in Matillion but don't see a suitable role, please email talent@matillion.com.

Matillion is an equal opportunity employer. We celebrate diversity and we are committed to creating an inclusive environment for all of our team. Matillion prohibits discrimination and harassment of any type. Matillion does not discriminate on the basis of race, colour, religion, age, sex, national origin, disability status, genetics, sexual orientation, gender identity or expression, or any other characteristic protected by law.