Senior Backend Software Engineer (Python, LLM)

As a Senior Backend Software Engineer on our LLM Services team, you’ll be at the forefront of integrating AI into healthcare delivery. Your tasks will include designing and implementing Python-based services that leverage Large Language Models to enhance our core healthcare platform. Working closely with our distributed engineering and product teams, you'll:

• Architect and build scalable micro-services that power our LLM-enabled features
• Collaborate with other Software, ML and Infrastructure engineers to optimize LLM integration and deployment
• Design and implement LLM orchestration services, including prompt management, model switching, and response streaming
• Build robust evaluation pipelines to measure and improve LLM output quality and consistency
• Develop scalable APIs for LLM-powered features, including context injection and retrieval augmentation
• Implement efficient caching and optimization strategies for LLM inference
• Create monitoring systems for tracking token usage, latency, and other LLM-specific metrics
• Work on prompt engineering and chain-of-thought implementations
• Ensure compliance with healthcare regulations while working with LLM outputs
• Participate in technical design discussions and code reviews
• Monitor and maintain services in production, including LLM Observability using tools like LangFuse

Requirements

• Strong proficiency in Python and experience building production-grade backend services
• Understanding of prompt engineering principles and LLM evaluation metrics
• Experience with vector databases (like Pinecone, Weaviate, or similar) for semantic search
• Familiarity with streaming architectures for real-time LLM responses
• Experience implementing rate limiting and failover strategies for API services
• Strong grasp of software engineering best practices, including testing, documentation, and version control
• Excellent problem-solving skills and attention to detail
• Strong written and verbal communication skills

You'd be a great fit if you have

• Experience fine-tuning or implementing RAG (Retrieval Augmented Generation) systems

• Experience with LLM frameworks such as LangGraph, LangChain, LlamaIndex, or similar orchestration tools
• Familiarity with different LLM providers (OpenAI, Anthropic, etc.) and their APIs
• Familiarity with the use of Open LLMs, either self-hosted or on through a third party provider, e.g. Amazon Bedrock
• Knowledge of LLM output validation and safety measures
• Experience with embeddings and semantic search implementations
• Background in prompt engineering or LLM evaluation metrics
• DevOps experience, particularly with containerised service deployment, GPU-enabled compute environments and scaling
• Knowledge of healthcare data privacy regulations and security best practices
• Experience working in distributed teams across multiple time zones

Location & Work Style

• All-remote position
• Flexible working hours. However due to need to work closely with engineers in Europe, we are not able to consider US applicants in the PST timezone
• Occasional late meetings to collaborate with US-based team members