Let’s get started
By clicking ‘Next’, I agree to the Terms of Service
and Privacy Policy
Jobs / Job page
Software Engineer, Core Engine image - Rise Careers
Job details

Software Engineer, Core Engine

About Eventual

Eventual is a data platform that helps data scientists and engineers build data applications across ETL, analytics and ML/AI.

OUR PRODUCT IS OPEN-SOURCE AND USED AT ENTERPRISE SCALE

Our distributed data engine Daft is open-sourced and runs on 800k CPU cores daily. This is more compute than Frontier, the world's largest supercomputer!

Daft is used at leading AI/ML companies such as Amazon, TogetherAI, EssentialAI, CloudKitchens and more. It makes ML/AI workloads easy and performant to run alongside traditional relational tabular workloads.

Today's “Big Data” data tooling (Spark, Trino, Snowflake) was built for a world of tabular data analytics. They do not generalize well to the needs of modern ML/AI data workloads. We built Daft to be the successor to these Big Data technologies along these core principles:

Python-native: Python is the native language of ML/AI and most of data engineering today

First-Class Local Development UX: Interactive development in a local Python notebook or script is where the magic happens

Multimodal Data Support: Modern workloads require support for operations on complex types such as long-form text, images, tensors and more

Heterogenous Compute (GPUs): GPUs are a requirement for workloads that perform model batch inference as part of the overall query

Key Responsibilities:

As a Software Engineer on the Core Engine team, you will build key capabilities for the Daft distributed data engine.

You will be working on core architectural design and implementation of various components in Daft including:

Planning/Query Optimizer: intelligently optimize users’ workloads with modern database techniques

Execution Engine: improve memory stability through the use of streaming computation and more efficient data structures

Distributed Scheduler: improve Daft’s resource utilization, task scheduling and fault tolerance

Storage: improve Daft integrations with modern data lake technologies such as Apache Parquet, Apache Iceberg and Delta Lake

Our goal is to build the world’s best open-source distributed query engine, becoming the leading framework for data engineering and analytics.

We are a young startup - so be prepared to wear many hats such as tinkering with infrastructure, talking to customers and participating heavily in the core design process of our product!

What we look for:

We are looking for a candidate with a strong foundation in systems programming and ideally experience with building distributed data systems or databases (e.g. Hadoop, Spark, Dask, Ray, BigQuery, PostgreSQL etc)

Our ideal candidate has:

3+ years of experience working with distributed data systems (query planning, optimizations, workload pipelining, scheduling, networking, fault tolerance etc)

Strong fundamentals in systems programming (e.g. C++, Rust, C) and Linux

Familiarity and experience with cloud technologies (e.g. AWS S3 etc)

Most importantly, we are looking for someone who works well in small, focused teams with fast iterations and lots of autonomy. If you are passionate, intellectually curious and excited to build the next generation of distributed data technologies, we want you on the team!


Benefits and Remote Work

We are believers in both having the flexibility of remote work but also the importance of in-person work, especially at the earliest stages of a startup. We have a flexible hybrid approach to in-person work with at least 3 days of in-person work typically from Monday - Wednesday at our office in San Francisco.

We believe in providing employees with best-in-class compensation and benefits including meal allowances, comprehensive health coverage including medical, dental, vision and more.


About the interview


INTRODUCTORY CALL [15M]

A short phone screen over video call with one of our co-founders for us to get acquainted, understand your aspirations and evaluate if there is a good fit in terms of the type of role you are looking for.

TECHNICAL PHONE SCREEN [1 HR]

A technical phone screen question over video call to understand your technical abilities.

TECHNICAL INTERVIEW PANEL [4 HR]

Technical interviews with the rest of the Eventual team with questions to further understand your technical strengths, weaknesses and experiences.


MEET THE TEAM

As many chats as necessary to get to know us - come have a coffee with our co-founders and existing team members to understand who we are and our goals, motivations and ambitions.

We look forward to meeting you!


WE'RE GROWING - COME GROW WITH US!

We are well funded by investors such as YCombinator, Caffeinated Capital, Array.vc and top angels in the valley from Databricks, Meta and Lyft.

Our team has deep expertise in high performance computing, big data technologies, cloud infrastructure and machine learning. Our team members have previously worked in top technology companies such as Amazon, Databricks, Tesla and Lyft.

We are looking for exceptional individuals with a passion for technology and a strong sense of intellectual curiosity.

If that sounds like you, please reach out even if you don't see a specific role listed that matches your skillsets - we'd love to chat!

Eventual Glassdoor Company Review
4.8 Glassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star icon Glassdoor star icon
Eventual DE&I Review
5.0 Glassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star iconGlassdoor star icon
CEO of Eventual
Eventual CEO photo
Unknown name
Approve of CEO

Average salary estimate

$125000 / YEARLY (est.)
min
max
$100000K
$150000K

If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.

What You Should Know About Software Engineer, Core Engine, Eventual

Join the innovative team at Eventual as a Software Engineer focused on our Core Engine! Located in the vibrant city of San Francisco, Eventual is all about enabling data scientists and engineers to create cutting-edge data applications seamlessly. You’ll be diving into the heart of our open-source distributed data engine, Daft, which is already making waves by running on a staggering 800k CPU cores daily. Your role will be pivotal in designing and implementing core components of Daft, including the Planning/Query Optimizer and Execution Engine. We're not just another tech company; we aim to revolutionize how ML/AI workloads are handled alongside traditional data processes. If you're passionate about systems programming and have a knack for building distributed data systems, you're the kind of engineer we're looking for. With your expertise, you'll help improve Daft's memory stability and resource utilization. This is a dynamic role where you’ll collaborate closely with our focused team, participate in design discussions, and even interact with customers. We value autonomy and quick iterations, and ideally, you have experience in C++, Rust, or similar languages as well as familiarity with cloud technologies. Plus, you’ll enjoy the flexibility of a hybrid work model, ensuring you have the best of both in-person collaboration and remote flexibility. If you’re excited to be part of a passionate team aiming to create the future of data technology, we’d love to hear from you!

Frequently Asked Questions (FAQs) for Software Engineer, Core Engine Role at Eventual
What are the key responsibilities for the Software Engineer, Core Engine at Eventual?

As a Software Engineer on the Core Engine team at Eventual, your key responsibilities will involve building essential capabilities for our Daft distributed data engine. You will engage in core architectural design and implementation for components like the Planning/Query Optimizer, Execution Engine, and storage solutions. This role also requires improving memory stability and ensuring efficient resource utilization and task scheduling. It's a chance to contribute meaningfully to creating one of the world's best open-source distributed query engines!

Join Rise to see the full answer
What qualifications are needed for the Software Engineer role at Eventual?

Eventual seeks a Software Engineer with a solid foundation in systems programming, ideally with over 3 years of experience in distributed data systems. Candidates should be proficient in languages such as C++, Rust, or similar, and have experience with cloud technologies like AWS S3. Familiarity with data systems and strong fundamentals in workload optimization and scheduling will be valuable in this role to ensure you're equipped to tackle the challenges of modern data workloads.

Join Rise to see the full answer
How does Eventual support remote work for the Software Engineer, Core Engine role?

Eventual believes in a balanced approach to work, combining the benefits of remote collaboration with essential in-person teamwork. For the Software Engineer position, a flexible hybrid model is in place, typically requiring at least three days of in-person attendance at our San Francisco office from Monday to Wednesday. This ensures effective collaboration while maintaining the flexibility needed for individual productivity.

Join Rise to see the full answer
What can I expect during the interview process for the Software Engineer at Eventual?

The interview process at Eventual for the Software Engineer, Core Engine role starts with a brief introductory call with one of our co-founders. This is followed by a technical phone screen to assess your abilities. Candidates then participate in a technical interview panel where further assessments and discussions take place. We also encourage candidates to meet with the team to learn more about our goals and culture, making the process enjoyable and insightful.

Join Rise to see the full answer
What sets Eventual apart as an employer for Software Engineers?

Eventual stands out as an employer by focusing on building cutting-edge technologies in the fast-evolving data landscape while fostering a collaborative culture. With backing from prominent investors and a team of experts from top tech companies, you'll have the opportunity to work on a meaningful product used by leading AI/ML companies. The environment encourages intellectual curiosity and autonomy, making it ideal for candidates eager to grow and innovate.

Join Rise to see the full answer
Common Interview Questions for Software Engineer, Core Engine
Can you describe your experience with distributed data systems?

In your response, focus on your specific experiences working with technologies such as Hadoop, Spark, or other distributed systems you've encountered. Mention the projects you worked on, emphasizing your role, the challenges you faced, and how you resolved them, showcasing your problem-solving skills and technical expertise.

Join Rise to see the full answer
What methods do you use for optimizing query performance?

When answering this, discuss various optimization techniques such as indexing, partitioning, and query rewriting. Provide examples from your past experiences where these methods led to significant performance improvements in a project, highlighting your understanding of database principles.

Join Rise to see the full answer
How do you ensure fault tolerance in distributed systems?

Talk about strategies you've employed to guarantee fault tolerance, such as retry mechanisms, data replication, and consistent state management. Drawing from specific scenarios where you implemented these strategies will demonstrate your practical knowledge in creating reliable systems.

Join Rise to see the full answer
What programming languages are you most comfortable with, and why?

Mention languages relevant to the Software Engineer, Core Engine role, such as C++, Rust, or Python. Discuss why you prefer these languages, focusing on their strengths in systems programming and how they relate to building high-performance applications, particularly in data systems.

Join Rise to see the full answer
Describe your experience with modern database technologies.

Here, emphasize your familiarity with various modern databases and data lakes, such as Apache Parquet and Delta Lake. Illustrate your experience in integrating these technologies into your projects and how they have improved data handling and querying efficiency.

Join Rise to see the full answer
How would you approach troubleshooting a performance issue in Daft?

Provide a structured approach for troubleshooting, starting with identifying performance bottlenecks using monitoring tools. Discuss how you would analyze logs and metrics, then test solutions iteratively while measuring the impact, showing your systematic problem-solving skills.

Join Rise to see the full answer
What role does cloud technology play in data engineering?

Outline how cloud platforms support scalability, flexibility, and resource optimization in data engineering. Provide examples of cloud services you've utilized, like AWS S3 or Azure Data Lake, and how they facilitated your projects or improved your workflows.

Join Rise to see the full answer
Can you explain the significance of the Execution Engine in a distributed data system?

Discuss the Execution Engine's role in managing the query execution phase, focusing on aspects such as resource allocation and execution strategy. Provide insights into why efficiency in this component is crucial for overall system performance and user experience.

Join Rise to see the full answer
How do you approach collaboration in small teams?

Describe your approach to effective communication and collaboration in a close-knit environment, emphasizing the importance of sharing ideas and feedback. Share examples of how you have successfully partnered with team members in the past to achieve a common goal, highlighting your interpersonal skills.

Join Rise to see the full answer
What excites you about working on open-source projects?

Discuss your passion for contributing to open-source projects and your belief in collaborative development. Share your experiences with open-source, whether through contributions, using tools, or understanding community dynamics, reflecting your enthusiasm for the transparency and innovation it fosters.

Join Rise to see the full answer
Similar Jobs
Photo of the Rise User
Ridgeline Hybrid San Ramon, CA; Reno, NV; New York, NY; Incline Village, NV
Posted yesterday
Photo of the Rise User
Posted 6 days ago
Photo of the Rise User
Leadtech Remote No location specified
Posted yesterday
Photo of the Rise User
Posted 14 days ago
Posted 13 days ago
Photo of the Rise User
Contentful Remote Dublin, Dublin, Ireland
Posted 2 days ago
MATCH
Calculating your matching score...
FUNDING
SENIORITY LEVEL REQUIREMENT
TEAM SIZE
EMPLOYMENT TYPE
Full-time, hybrid
DATE POSTED
January 9, 2025

Subscribe to Rise newsletter

Risa star 🔮 Hi, I'm Risa! Your AI
Career Copilot
Want to see a list of jobs tailored to
you, just ask me below!