Databricks Admin

The Azure Databricks Administrator will be responsible for providing technical expertise in deploying applications on Azure Databricks platform.

The Databricks Administrator must be self-motivated and apply knowledge of Azure to drive solutions, support the development team, and create documentation to support and describe technical solutions.

An ideal candidate will have deep technical Azure platform expertise and demonstrate high standards of accountability, customer focus, and quality.

Requirements

Job Description:

• 
  Databricks access control
• 
  Databricks Cost Management
• 
  Databricks Performance Management.
• 
  Azure Application Gateways
• 
  Databricks Backup (IaC based)
• 
  Azure virtual network Databricks Storage (DBFS)o Azure Storage (ADLS)
• 
  Databricks SQL – Delta Live Table
• 
  Disaster Recovery Deploy and maintain PaaS & SaaS Databricks platform for Machine Learning (ML) development, QA, staging, and production environments hosted on Azure cloud infrastructure.

Understanding of a multitude of Azure resources and services such as containers, virtual machines, storage, firewalls, network security groups, web application firewalls, vnets, log analytics, sentinel, azure monitor, alerting, metrics, Data Factory, Databricks, Azure AD, Data Catalog, Web Apps, Functions,

Application Service Environments, Key Vaults, Azure Backups.Manage Databricks environments in line with security guidelines.

Develop Infrastructure as Code(IaC) to automate databricks environment deployments.

Develop a library of scripts to automate databricks MLOps environments.Identify and generate solutions to current problems in the organization.

Aid developers and DevOps teams with implementing automation scripts that follow the company standards.

Collaborating with DevOps team members as well as all of engineering ensures that solutions exhibit high levels of performance, security, scalability, maintainability, appropriate reusability, and reliability upon deployment.

Partner with Cloud solution architect and development team to fully understand product requirements and configure the Databricks environments which meet these requirements.

Evaluate and automate the scaling and capacity requirements within Azure environments. Create custom scripts to automate support processes where applicable.

Manage Databricks infrastructure and collaborate with various teams to ensure it integrates seamlessly with all components including network, security, messaging, and remote access.

Troubleshoot and resolve Databricks environment performance issues, connectivity issues, security issues.

Research, adopt and relate new technologies to solve issues that arise while adhering to company security and standards.

Solid experience in the administration and performance tuning of Databricks is required for success at this position.

Working knowledge of Azure Active Directory, Snowflake and Azure Cloud components is a PLUS.

Solid scripting skills is required for success at this position.

Experience in working with formal Change Control Systems. Mentor, coach, and share knowledge with other team members throughout the organization.

Other duties as assigned.

5+ years’ experience in Databricks administration

Benefits

• Competitive salaries
  
• Medical, Dental, Vision Insurance
  
• Disability, Life & AD&D Insurance
  
• 401K With Generous Company Match
  
• Paid Vacation and Personal Leave
  
• Pre-Paid Commute Options
  
• Employee Referral Bonuses
  
• Performance Based Bonuses
  
• Flexible Work Options & Fun Culture
  
• Continuing Education Reimbursements
  
• In-House Technology Training