Data Engineer

• The Data Governance/Data Quality Subject Matter Expert (SME) is a professional responsible for designing, implementing, and managing the client’s data governance strategy and ensuring high data quality standards across data-centric operations and decision-making.
• The SME is in charge of developing and maintaining robust data pipelines, ensuring data observability, managing data within the Microsoft Azure environment, and implementing data governance principles in alignment with industry best practices and regulatory standards.

• Data Governance & Quality: This includes developing and implementing data governance policies and procedures, monitoring data quality, managing data within the Microsoft Azure environment, and ensuring compliance with data privacy regulations.
• Data Pipeline & Workflow Development: Designing and implementing efficient and secure data pipelines in Azure Data Factory and workflows, ensuring optimal performance and incorporating data quality principals.
• Data Integration & Observability: Designing data integration strategies, integrating data from multiple sources, ensuring data quality and observability through tools like Azure Synapse.
• Data Security: Defining and implementing data security policies and procedures, including access control and data encryption within the Azure environment.
• Performance Tuning: Optimizing data access, indexing, and query performance within Azure Data Factory to ensure scalable and reliable data infrastructure.
• Monitoring & Troubleshooting: Monitoring data systems, troubleshooting issues, identifying performance bottlenecks, and collaborating with team members to resolve problems.
• Collaboration & Communication: Working closely with stakeholders to understand their data needs and develop solutions adhering to industry best practices and the needs of the business.
• Automation & Tooling: Automating repetitive tasks and maintaining tools necessary for data processing and storage within the Azure environment.

• A Bachelor's degree in a related field such as computer science, mathematics, engineering, or information systems.
• 3+ years of hands-on experience programming in SQL.
• 2+ years of experience building and maintaining automated data pipelines and data assets using batch and/or streaming processes within the Microsoft Azure environment.
• Experience in Data Governance OR Data Quality Management is a must.

Competencies:

• Expertise in Microsoft Azure data tools, with a focus on Azure Data Factory and Azure Synapse for data processing, storage, and analysis.
• Strong SQL skills and knowledge of data warehousing concepts and ETL (Extract, Transform, Load) processes within the Azure environment.
• Proficiency in programming languages such as Python, and experience with big data technologies such as Hadoop, Spark, and Kafka.
• Ability to design, build, and maintain data pipelines, with a particular focus on Azure Data Factory.
• Strong analytical and problem-solving skills, with the ability to identify and resolve data-related issues effectively.
• Experience with data security and privacy regulations, and understanding of HIPAA compliance.
• Strong experience with data governance, data quality, and data observability principles.
• Excellent written and verbal communication skills, ability to explain complex technical concepts to non-technical stakeholders.
• Excellent collaboration and teamwork skills, with the ability to work in a team involving client data engineers, data scientists, and data analysts.
• Strong attention to detail and commitment to ensuring high data quality.
• Continuous learning mindset and ability to stay up-to-date with new developments and technologies in data governance and data quality
• Works independently or well within a team
• Wants to continuously grow knowledge base and skill set
• Collaborative, consultative mindset
• Works well in a fast paced environment
• Strong technical background
• Deep knowledge and curiosity about technology and systems
• Agile mindset