Data Engineer

OUR TEAM:
We are the Data Science and Integration team. We are made up of different specialists who play multiple roles: data scientists, data architects, data engineers, business analysts, and database administrators. We provide our expertise in all things data to Promega. We lend our expertise to Integrating data, moving data, and providing insights into data. Our diverse internal clientele includes all area of Promega’s business. We strive to answer the questions our partners haven’t even asked us yet.

YOUR ROLE:
From Madison, WI or anywhere remotely, the Data Engineer provides oversight and development around the implementation of a modern data platform solution. You will design, develop, modify and support data solutions in Azure and on-premises environments. You are also a key contributor in modeling, developing, enhancing, and supporting curated data models.

JOB OBJECTIVE: The Data Engineer designs, develops, enhances, and supports data solutions with an Azure/cloud-first architecture approach.

1. Design, develop, modify, and support data solutions in Azure and on-premises environments.

2. Design and develop scalable, efficient (internal and external) data pipeline processes to handle data ingestion, cleansing, transformation, integration, and validation.

3. Model, develop, enhance, and support curated data models.

4. Ensure standards are met so that solutions deployed have technical integrity and stability.

5. Design and develop automated Data Quality solutions that ensures the integrity of a modern data platform.

6. Document production software, runbooks, and solution architectures.

7. Develop, communicate, and present solution architectures to both technology and business stakeholders.

8. Support critical strategic initiatives and architect simple to complex data engineering projects.

9. Collaborate with IT peers to understand the Data Landscape across the organization.

10. Collaborate with Data Scientists, Data Analysts and Business Stakeholders to deliver curated data models.

11. Build cross-functional relationships with Business Stakeholders, Architects, Data Scientists, BI Managers, and Business Partners to understand data needs and help projects and agile teams deliver on those needs.

12. Deliver collaborative products that align with organizational strategies.

13. Mentor junior level engineers and cross team peers in all levels of data engineering.

14. Work with other IT teams to write and maintain technical and en-user documentation and provide training.

15. Provide off-hours (24x7) support for Promega’s Data Systems as required.

16. Stay up on current technologies and industry trends.

17. Provide other duties as assigned.

18. Demonstrate inclusion through their own words and actions and be accountable for a safe workspace. Act with kindness, curiosity and respect for others.

19. Embrace and be open to incorporating Promega’s 6 Emotional & Social Intelligence (ESI) core principles in daily work.

20. Understand and comply with ethical, legal and regulatory requirements applicable to our business.

1. Bachelor’s degree in computer science or relevant science discipline.

2. 5+ years’ experience as a Data Engineer or equivalent role.

3. Working knowledge of Data Engineering tools like Databricks, Synapse, Spark, PySpark, Scala, Data Factory, SSIS, Informatica.

4. Comfortable in building effective analytical tools that utilize the data pipeline to provide actionable insights into data synchronization and operational efficiency of the data repositories.

5. Experience in designing database models to store structured & unstructured data efficiently and in creating effective data tools for analytics experts.

6. Advanced knowledge in relational databases, like MSSQL, MySQL, Oracle, etc.

7. Knowledge of and ability to implement CI/CD best practices especially as they relate to data engineering.

8. Experience in defining and setting up data architecture to support data warehousing and business intelligence.

9. Working knowledge on enterprise data lakes, data analytics, reporting, in-memory data handling, enterprise integration tools, etc.

10. Having an owner's mindset and experience working in a fast-paced and demanding environment.

11. Ability to work in an Agile development environment.

1. Microsoft Azure experience, especially in the data engineering space.

2. Knowledge of no-SQL databases, Azure cosmos DB, MongoDB, Elastic Search, etc.

3. Knowledge on enterprise data lakes, delta lake, data analytics, in-memory data handling, enterprise integration tools, etc.

4. Knowledge of open source tools like NiFi, Kafka, Airflow, etc. specifically on the Azure cloud platform.

5. Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.

6. Proven ability in working in an agile team environment, anticipating, and meeting team needs, working with little direction, and gaining cooperation from others.

7. Experience and/or training in designing, developing, supporting, and maintaining business application software, including extensive end-user contact.

8. Ability to work in a fast-paced, environment with frequent changes in priorities and assignments.

1. Ability to use a personal computer and manipulate software to accomplish required tasks as outlined above daily including keyboarding skills.

2. Ability to distinguish different colors and shades, tints, and tones of color which is essential for graphical design work.

3. Availability to occasionally work overtime hours to complete assignments during peak work-load periods.

Diversity is important at Promega. We are proud to be an Equal Opportunity Employer, and make employment decisions without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, disability, or any other protected class.