Data Engineer

Overview

As a Data Engineer at Inception, you will be responsible for building and maintaining the infrastructure that supports data collection, processing, and storage. Reporting to the Lead Data Engineer or Principal Data Engineering, you will work closely with data scientists, analysts, and other stakeholders to ensure that data systems are reliable, scalable, and secure. Your work will be crucial in enabling data-driven decision-making across the organization.

The opportunity

The Data Engineer is a key technical role focused on developing and optimizing the company's data infrastructure. This role involves designing and implementing data pipelines, ensuring data quality, and collaborating with cross-functional teams to support various data initiatives. The ideal candidate will have strong technical skills in data engineering, experience with data architectures, and a passion for working with data.

Inception is the UAE’s national-scale enabler in AI Research and Development. Partnering with Microsoft's AI SaaS, we offer domain-specific Agentic AI Orchestrator platforms utilizing reasoning agents for precise and cost-effective services. Our focus includes AI incubation, IP creation, applied AI R&D, and AI investment products. By creating models tailored to specific domains and languages, we ensure superior accuracy and efficiency. Collaborating with top universities and industry giants to drive significant advancements in AI technology within the region.

Responsibilities

As a Data Engineer, you will be responsible for developing and maintaining data systems to support the company’s strategic goals. Your role will encompass a range of activities focused on data pipeline development, data quality, and cross-functional collaboration.

·       Data Pipeline Architecture and Development

o   Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics.

·       Data Integration

o   Work closely with data scientists, ML engineers, and stakeholders to ensure that data is accessible, consistent, and reliable for ongoing projects.

·       API and Data Services

o   Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed.

·       Data Storage

o   Manage and optimize data storage solutions for both structured and unstructured data, where structured data includes relational databases and unstructured data includes Text Image Audio and Video, Search Engines like Elasticsearch and NoSQL databases, to support the requirements of machine learning models.

o   Understand data engines and structure to effectively design solutions for transactional, analytics, and search purposes.

·       Data Quality and Governance

o   Implement processes to monitor data quality and ensure production data is always accurate and available for key stakeholders.

·       Collaboration and Support

o   Collaborate with ML engineers to assist in data-related technical issues and provide architectural guidance and solutions.

·       Security and Compliance

o   Ensure compliance with data security and privacy policies.

·       Documentation

o   Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams.

Qualifications

Skills and attributes for success

o   Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.

o   3+ years of experience in a Data Engineering role.

o   Programming languages like Python, Java, and Scala and managing huge scale data potentially Terabyte to Petabyte.

o   Hands-on experience with big data technologies like Hadoop, Spark and Flink.

o   Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar.

o   Strong understanding of data warehousing concepts, ETL processes, and data modeling.

o   Experience with API development and integration with data services.

o   Experience with cloud platforms like Azure.

o   Knowledge in DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes.

o   Experience with real-time / streaming data processing.

Technical stack

o   Programming Languages: Python, Java, Scala, SQL, Bash

o   Big Data Technologies: Hadoop, Spark, Flink

o   Unstructured Data: Text, Image, Audio & Video

o   Databases: MySQL, PostgreSQL, MongoDB, Cassandra, HBase,  Redis

o   Cloud Platforms: Azure

o   API Development: RESTful APIs, GraphQL, OpenAPI

o   Data Services: Kafka, RabbitMQ

o   Containers: Docker, Kubernetes

What we look for

If you are a performance-driven, inquisitive mind with the agility to adapt to ambiguity, you will fit right in. You should be eager to explore opportunities to build meaningful collaborations with stakeholders and aspire to create unique customer-centric solutions. Bias for action and a passion to conquer new frontiers in the AI space is at the heart of the Inception community.

What working at Inception offers

Culture: An open, diverse and inclusive environment with a global vision that encourages personal growth and focuses on ground-breaking, industry-first innovations.

Career: Outstanding learning, development & growth opportunities via structured training programs and innovative, high-tech projects.

Work-Life: A hybrid work policy to strike the perfect balance between office and home.

Rewards: A competitive remuneration package with a host of perks including healthcare, education support, leave benefits and more.

If you can confidently demonstrate that you meet the criteria above, please contact us as soon as possible.