Job details

Senior Data Engineer

Get a free resume review

This role is for one of the Weekday's clients

We are looking for a Senior Data Engineer to design, build, and scale our data platform, enabling efficient data processing and analytics across the organization. In this role, you will develop and optimize batch and real-time data pipelines, ensuring high-performance data infrastructure that drives business insights. This is an excellent opportunity for someone passionate about modern data technologies, cloud-native environments, and scalable data systems.

Key Responsibilities:

Data Platform Development:

Design, develop, and maintain a scalable and reliable data platform that supports batch and real-time data processing.
Optimize the platform for cost efficiency, performance, and scalability to meet growing business needs.

Data Pipeline Management:

Build and manage ETL/ELT pipelines for data ingestion, transformation, and distribution across teams.
Ensure pipelines are resilient, high-performing, and scalable.

Real-Time Data Processing:

Implement low-latency, high-throughput streaming solutions using technologies like Apache Kafka and Apache Flink.

Data Storage Optimization:

Design and optimize data lakes and data warehouses for efficient data retrieval and cost management in cloud environments.
Work with modern table formats like Hudi, Iceberg, or Delta Lake.

System Reliability and Monitoring:

Implement monitoring solutions and data quality frameworks to ensure data accuracy, reliability, and security.
Identify and resolve performance bottlenecks in data pipelines.

Cross-Team Collaboration:

Work closely with data scientists, analysts, and product teams to develop scalable data solutions that meet evolving business needs.

Data Security & Compliance:

Implement best practices for data security, compliance, and privacy to ensure integrity across the platform.

Technical Leadership:

Provide mentorship and technical guidance to junior engineers.
Drive best practices in system design, coding, and deployment.

Automation & Infrastructure as Code (IaC):

Automate data infrastructure management using Terraform or Ansible for consistency and efficiency.

Required Qualifications:

6+ years of experience in data engineering with a focus on scalable data platforms.
Strong experience designing and implementing batch and real-time data pipelines in cloud environments (AWS preferred).
Expertise in distributed data processing frameworks (Apache Spark, Presto) and streaming technologies (Kafka, Flink).
Hands-on experience with data lake architectures and modern table formats (Hudi, Iceberg, or Delta Lake).
Strong programming skills in Python and/or Java, with experience building scalable and maintainable data solutions.
Proficiency in SQL and/or NoSQL databases.
Experience managing cloud infrastructure using Infrastructure as Code (Terraform, Ansible).
Performance tuning and optimization of large-scale data systems.
Strong problem-solving skills with the ability to prioritize tasks and adapt to evolving project needs.
Excellent communication and leadership skills, with a track record of mentoring junior engineers.
Bachelor’s/Master’s degree in Computer Science, Engineering, or a related field.

Preferred Qualifications:

Experience with DataOps and CI/CD pipelines for data engineering.
Hands-on experience building data platforms from scratch.
Familiarity with Kubernetes is a plus.
Startup experience, with the ability to innovate and adapt in a fast-paced environment.

Average salary estimate

$125000 / YEARLY (est.)

min

max

$100000K

$150000K

If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.

What You Should Know About Senior Data Engineer, Weekday

Are you ready to take your data engineering skills to the next level? Join a leading client of Weekday as a Senior Data Engineer, where your expertise will shape the future of data processing and analytics. In this exciting role, you'll have the opportunity to design, build, and scale a robust data platform that ensures high-performance infrastructure for driving business insights. Your day-to-day will involve optimizing batch and real-time data pipelines, implementing modern technologies like Apache Kafka and Apache Flink, and collaborating with data scientists and analysts to create solutions that meet evolving business needs. We’re looking for someone who’s passionate about modern data technologies and fostering best practices, mentoring junior engineers while also balancing hands-on technical work. If you're an experienced data engineer with a strong background in cloud-native environments and scalable data systems, this is your chance to make a significant impact in a dynamic organization. We offer a collaborative environment where innovation is encouraged and your contributions are valued. Step into this exciting role and let your skills shine!

Frequently Asked Questions (FAQs) for Senior Data Engineer Role at Weekday

What are the key responsibilities of a Senior Data Engineer at Weekday?

As a Senior Data Engineer at Weekday, you will be responsible for designing, building, and maintaining a scalable and reliable data platform that supports both batch and real-time data processing. You'll manage ETL/ELT pipelines, ensuring resilience and high performance. Moreover, implementing low-latency streaming solutions, optimizing data lakes and warehouses, and collaborating with cross-functional teams will be crucial parts of your role.