Data Engineer - job 2 of 3

Experience Required:

• 3+ years of experience in data engineering or a related field.
• Expertise in MLOps (Machine Learning Operations).
• Preferred experience or certification in DataIKU.

Key Skills:

• Strong knowledge of data engineering principles and best practices.
• Proficiency in MLOps frameworks and tools for the development, deployment, and monitoring of machine learning models.
• Experience with cloud platforms (AWS, Azure, Google Cloud) and their data engineering services.
• Familiarity with data integration, ETL processes, and data pipelines.
• Strong coding skills in Python, SQL, and other relevant programming languages.
• Experience with big data technologies (Hadoop, Spark, etc.).
• Familiarity with containerization technologies such as Docker and Kubernetes.

Responsibilities:

1. Data Pipeline Development:
• Design, build, and maintain scalable data pipelines for the collection, processing, and storage of large datasets.
• Ensure efficient data flow across various systems, ensuring the integrity and availability of data for analytics and machine learning models.
2. MLOps Implementation:
• Implement MLOps best practices for model development, deployment, and monitoring in production environments.
• Collaborate with data scientists and machine learning teams to automate workflows and ensure efficient model lifecycle management.
3. Data Infrastructure Management:
• Manage and optimize data infrastructure on cloud platforms to ensure efficient data processing, storage, and retrieval.
• Implement and maintain data warehouses, data lakes, and other data storage solutions.
4. Collaboration and Support:
• Collaborate with cross-functional teams including data scientists, analysts, and product teams to understand business requirements and provide data solutions.
• Provide ongoing support for data pipelines, monitor performance, and resolve any data-related issues.
5. Tool and Technology Expertise:
• Stay updated on the latest trends in data engineering and machine learning technologies.
• Leverage DataIKU (or similar tools) to streamline data workflows, automate processes, and enhance overall efficiency.

Qualifications:

• Bachelor's or Master’s degree in Computer Science, Engineering, Data Science, or related field.
• 3+ years of experience in data engineering, with a focus on MLOps and data pipeline development.
• Certification or hands-on experience with DataIKU is highly preferred.
• Strong problem-solving skills and the ability to work independently and in a collaborative team environment.
• Proficiency in Python, SQL, and cloud technologies (AWS, Azure, or Google Cloud).