Azure Data Engineering - Architect

Responsibilities:

• Designs, develops, optimizes, and maintains data architecture and pipelines that adhere to Data Pipeline (ELT) principles and business goals.

• Solves complex data problems to deliver insights that help the business achieve its goals.

• Creates data products for engineers, analysts, and data scientist team members to accelerate their productivity.

• Engineers effective features for modeling in close collaboration with data scientists and businesses.

• Leads the evaluation, implementation, and deployment of emerging tools and processes for analytics data engineering to improve productivity and quality.

• Partners with machine learning engineers, BI, and solutions architects to develop technical architectures for strategic enterprise projects and initiatives.

• Bachelor’s degree in computer science, statistics, engineering, or a related field.
• 10-15 years of experience required.
• Experience in PySpark is required for this position, with strong knowledge of Microsoft Fabric being a plus.
• Experience designing and maintaining data warehouses and/or data lakes with big data technologies such as Spark/Databricks or distributed databases like Redshift and Snowflake.
• Experience in housing, accessing, and transforming data in various relational databases.
• Experience building data pipelines and deploying/maintaining them following modern Data Engineering best practices (e.g., DBT, Airflow, Spark, Python OSS Data Ecosystem).
• Knowledge of Software Engineering fundamentals and software development tooling (e.g., Git, CI/CD, JIRA), with familiarity with the Linux operating system and the Bash/Z shell.
• Experience with cloud database technologies (e.g., Azure) and developing solutions on cloud computing services and infrastructure in the data and analytics space.
• Basic familiarity with BI tools (e.g., Alteryx, Tableau, Power BI, Looker).
• Expertise in ELT and data analysis, primarily SQL.