Data Engineer

Description

Company Summary

CRISP Shared Services (CSS) serves as the technology partner for health information exchange (HIE) affiliates in Maryland, DC, West Virginia, Connecticut, Alaska, and Virginia. CSS facilitates the electronic transfer of clinical information between disparate information systems in order to positively impact the healthcare system through improved patient care and outcomes, reduced costs, and data-driven understanding of regional public health concerns.

Job Summary

The Data Engineer will be responsible for supporting HIT activities in the Baltimore/Washington region. The successful candidate will work to design, develop, and maintain a data analytics platform solution. CRISP Shared Services is currently running an Azure Data Lake 2 / Azure Data Factory / Spark / Databricks infrastructure with approximately 300 production use cases that consolidate several streams of healthcare data. This role will primarily focus on the creation of new use cases through Databricks programming including communication, technical proficiency, collaboration, and willingness to learn new things. This role assists a senior data engineer to implement new ingestion pipelines and troubleshoot production issues.

This position will have a base salary.

This position requires the candidate to be in the Columbia office 1 day a week.

Requirements

Essential Duties and Responsibilities

Include the following. Other duties may be assigned.

• Design, build, and maintain large, complex data processing pipelines that meet business requirements
• Ensure the quality of deliverables by developing automated controls and performing unit, integration, and user acceptance testing
• Develop scalable and re-usable frameworks for ingestion and transformation of large data sets
• Troubleshoot and perform root cause analysis on data pipeline issues
• Work with source system owners and business owners to incorporate business changes into data pipeline
• Create comprehensive documentation of data workflows, processes, and infrastructure

Qualifications

To perform this job successfully, the incumbent must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

• Advanced programming skills in PySpark, Python, Spark SQL
• Thorough understanding of Data Lakes, raw/enriched/curated layer concepts, and ETL within Azure framework
• Proven development experience in building complex data pipelines for lakehouse/data warehouses using Agile methodology
• Experience in architecture, design, and implementation using Databricks and Azure Data Factory

Experience and/or Educations Requirements

• 4+ year college degree (required)
• 4+ years of working with data integration (required)
• 2+ years of advanced programming in PySpark
• Experience working with Databricks (preferred)

Compensation and Benefits Package 

• Targeted base salary: 95k – 115k per year, based on experience and qualifications, plus benefits and bonuses.
• Benefits include the following but are not limited to health, dental, vision, life and disability insurance. CSS also has a 403(b) plan with discretionary employer match and contributions. You will be eligible for 20 days of Paid Time Off per year (accrued in accordance with CSS’s policies). PTO is in addition to 8 company designated holidays. If you are not PTO eligible, you may be eligible for applicable state and/or local sick and safe leave in accordance with the provisions of any applicable state and/or local sick and safe leave laws.

www.crispsharedservices.org  

www.crisphealth.org  

CRISP Shared Services, Inc. is an equal opportunity employer. This means that CRISP Shared Services Inc. is dedicated to providing equal employment opportunities to all employees and applicants for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, age, marital status, genetic information, disability, military or veteran status, or any other status protected by federal, state, or local law. We celebrate diversity and are committed to creating an inclusive environment for all team members.