Engineering Manager, Data Acquisition (Foundations)

About the team:

The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support all training and scaling across the research organization. Our team manages web crawling and GPTBot services and works closely with the Data Processing, Architecture, and Scaling teams. We are seeking a Senior Engineering Manager to lead our Data Acquisition team, overseeing engineering initiatives and ensuring effective collaboration across sub-teams.

What you will work on:

• Lead and manage a team, driving engineering projects related to web crawling, data ingestion, and search capabilities.

• Oversee the architecture, development, and deployment of scalable distributed systems capable of handling petabytes of data.

• Collaborate closely with cross-functional teams, including Data Processing, Architecture, Scaling, and Legal, ensuring seamless integration, data flow, and system compliance with data privacy regulations.

• Provide technical leadership, mentoring, and career development to engineers within the team, fostering an inclusive and high-performing environment.

• Develop and maintain long-term strategic planning for data acquisition needs, identifying new opportunities for data collection and processing improvements.

• Partner with senior leadership to align team priorities with broader organizational goals and ensure successful execution of team initiatives.

• Ensure high reliability, scalability, and performance of data acquisition services, optimizing existing processes and introducing new technologies where appropriate.

• Advocate for best practices in software engineering, including robust testing, system monitoring, and continuous improvement of team processes.

• Oversee routine system checks and monitoring for backend services deployed in a Kubernetes Infrastructure-as-Code environment.

Qualifications:

• 10+ years of industry experience in software engineering, with a focus on large-scale distributed systems and data processing.

• 4+ years of experience in managing and leading engineering teams, with a proven ability to mentor and grow engineering talent.

• Strong expertise in building and scaling large web crawlers, data pipelines, and ingestion systems.

• Experience working with Kubernetes and Infrastructure-as-Code concepts.

• Demonstrated ability to drive complex, cross-team engineering projects and lead collaboration across different functions.

• Strong technical and strategic vision, with the ability to translate broad objectives into actionable engineering plans.

• Excellent communication skills, with the ability to effectively interact with technical and non-technical stakeholders.

• A passion for innovation and the ability to foster a culture of learning and continuous improvement.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. 

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status. 

OpenAI Affirmative Action and Equal Employment Opportunity Policy Statement

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.