Sr. Software Engineer (Query Optimization)
Who We Are:
Materialize is the streaming SQL database company that makes it easy for any developer or analyst to understand streaming data, answer complex questions and build intelligent applications using standard technology. Whether it's delivering personalized experiences, accurately identifying fraud, building predictive AI, or discovering new business opportunities, the ability to run complex queries on multiple streams of data and keep their answers up to date is critical to making better decisions about the changing world around us.
We are focused on bottom-up developer adoption, our core software is free to use and source available, our business model is a cloud product that handles management of Materialize and lets businesses focus on building value. We have a vibrant developer community on Slack and as much as possible we work in public on GitHub.
The Materialize team includes engineers who were early employees of Cockroach Labs, Ververica, and Stripe. Our work builds on top of Timely Dataflow and Differential Dataflow, both created by our co-founder Frank McSherry, a world-leading computer scientist with decades of award-winning research in all aspects of data.
Investors:
Kleiner Perkins, Redpoint Ventures and Lightspeed Venture Partners.
About the Role:
We are looking for seasoned engineers with experience in query optimization who are excited to be part of a small and highly autonomous team that is focused on Materialize's query optimization pipeline.
Materialize is built on top of Differential Dataflow which is an award winning project with strong theoretical foundations. Differential Dataflow models computations as dataflows and allows Materialize to quickly respond to arbitrary changes in input data while maintaining consistency for materialized views.
The optimization team is responsible for the generation of robust and efficient dataflow plans from SQL queries by applying proven optimization techniques and exploring new ones that are tailored to the dataflow environment.
Joining this team will give you the opportunity to play a crucial role for the overall success of Materialize as your work will directly impact our performance characteristics and the correctness guarantees we make to our customers.
Responsibilities:
- Work as part of the query optimization team on designing, implementing, and maintaining substantial parts of Materialize's query optimization pipeline in Rust.
- Evolve the query optimizer to generate robust and efficient plans from SQL queries while allowing the team to productively iterate on new and existing optimizations.
- Explore how optimization on top of a continuously running dataflow engine differs from more traditional systems and identify opportunities to adapt existing optimization techniques to this environment as well as develop new techniques.
- Work with non-engineering teams such as product management to develop an understanding of real-world customer usage and help to adapt Materialize to customer needs.
The following links provide a small sample of the kind of work the team has been doing in the past:
- https://materialize.com/docs/sql/explain/
- https://materialize.com/joins-in-materialize/
- https://materialize.com/temporal-filters/
About You:
- 3+ years of experience with software engineering or in academia focused on optimization in the context of database management systems (SQL, NoSQL, NewSQL), compilers, or related areas.
- Solid programming fundamentals (e.g. C++, Java, Python) and interest in learning Rust.
- Track record of learning new technologies and concepts quickly.
- Ability to work both autonomously and collaboratively, as needed.
- Strong written and verbal communications skills.
- Strong working knowledge of computer science fundamentals, equivalent to a B.S., M.S., or Ph.D in Computer Science. Timely and Differential Dataflow are built upon years of academic and industrial research, and you'll need to become familiar with the research areas.
Bonus Points:
- Knowledge of stream processing.
- Experience with Rust.
We understand it takes a diverse team of highly intelligent, passionate, curious, and creative people to develop the exceptional product we are building. Our dynamic team has incredible perspectives to share, just as we know you do, and we take great pride in being an equal opportunity employer.
By
Materialize
The Cloud Database for Fast-Changing Data. We put a streaming engine in a database, so your team can build real-time data products without the cost, complexity, and development time of stream processing.
14 jobs
Subscribe to Rise newsletter
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
Sign up for our weekly
newsletter of fresh jobs
