Lead Data Engineer

***Must be located in the SF Bay Area or your application will not be considered***

Our client is a venture-backed SaaS startup that recently secured Series A funding. They're solving a multi-billion dollar problem of data quality for the entire marketing industry. Their platform enables major brands and publishers to optimize consumer data quality, improving marketing ROI. In a fast-paced and collaborative environment, they are committed to excellence and innovation.

Key Responsibilities:

• Design and maintain scalable data pipelines using Spark and AWS EMR for processing terabytes of consumer data

• Write, test, and optimize custom Scala code for ETL workflows
• Deploy and manage ETL processes in AWS cloud environments using Airflow
• Automate delivery of structured data to enterprise clients via Snowflake and Databricks
• Collaborate with the Data Science team to optimize data infrastructure
• Contribute to data modeling efforts using SQL for efficient large-scale data management
• Create monitoring tools and KPIs using Tableau to ensure data pipeline health
• Maintain comprehensive documentation for internal teams and external clients

Tech Stack: AWS (EMR, EC2, S3, Athena, Sagemaker), Spark, DBT, Snowflake, Databricks, Airflow, Terraform, Github, Tableau

Programming Languages: Scala, Python, SQL, and Bash

• 5-7 years of relevant work experience (3-5 years considered)

• Strong SQL and Scala skills
• Experience with cloud computing tools (e.g., Spark, AWS EMR, Snowflake, Databricks)
• Proficiency in data modeling and distributed data processing
• Excellent communication skills, including explaining complex concepts to non-technical stakeholders

An ideal candidate will:

• Thrives in a fast-paced startup, embracing high-impact responsibilities with minimal oversight
• Strong communicator, adept at explaining complex data concepts to diverse stakeholders
• Passionate about solving data quality challenges in marketing, with relevant industry experience
• Committed to long-term growth, excited by potential for ownership and future team leadership

Compensation:

• Salary range: $160,000 - $180,000
• Stock options package
• Full health benefits and 401k

Work Environment:

• Hybrid model: 3 days in office, 2 days remote, on average
• Small, high-impact team with plans for growth
• Report directly to the Head of Data Science