Data Engineer

Job Summary• Augment and maintain the existing repositories and data structures within AWS (used to process and store large amounts of data from unrelated sources)• Experience with several formats and means for data ingestion. Including, data types (structured, semi-structured, and unstructured), and sources (on premise, and in the cloud) using the most appropriate techniques in each case• Continue to expand and enhance the model, utilizing best practices, in regards to the organization of data and the various relationships• Optimize existing and future models for fast and scalable queries (while maintaining performance and related price thresholds)• Work with the team to define, construct, and maintain self-service dashboards for the Business and Advanced Analytics teams within PowerBI• Implement scalable and flexible, high performance data pipelines for AWS to support analytics• Develop and maintain data maps and their relationships• Generate associated technical documentation including follow-up reports• Work with Data Governance to implement quality rules and data governance measures (data dictionary, metadata, traceability, ...)• Propose improvements and actions based on provided results• Communicate results effectively with required teamsKnowledge, Skills and Abilities:• Bachelor's Degree with 6+ years of experience.• Advanced knowledge and experience using Python, Airflow, Spark, AWS, and Snowflake• Database architectures: SQL, NoSQL, graph databases• CI/CD and Orchestration: Jira, Jenkins, Bit Bucket, Terraform, and Airflow• Past experience with data modeling tools, ETL tools (e.g. Informatica Power Center)• Computer languages, data query and transformation tools: AWS Athena, Jupyter notebooks, Spark, Pyspark, Python, and EMR Studio• Algorithm analysis (for working with our Data Scientists)• Understanding of multidimensional modeling for quantitative and fact related data storage• OS: Linux, and MS Windows• Code IDE: Microsoft Visual Code, Jupyter notebooks• Artificial intelligence, machine learning, and deep learning are a plus