• Manage and maintain the underlying data infrastructure, which typically exists on a major cloud platform like AWS, Azure, or GCP
• Implement data governance practices, including data lineage, metadata management, and access controls integrating with other 3rd party products like Immuta,Protegrity for tokenization etc
• Adhere to software engineering best practices, including participating in code reviews and CI/CD (Continuous Integration/Continuous Deployment) automation
• Stay up to date with the latest trends and technologies in data engineering and the Databricks ecosystem
• Design and build robust ETL (Extract, Transform, Load) and ELT workflows to ingest, transform, and load structured and unstructured data from various sources
• Utilize Databricks features like Delta Live Tables and Databricks Workflows to orchestrate and manage complex data processes
• Optimize and tune Apache Spark jobs for performance and cost efficiency on large datasets
• Implement and enforce data security, access control, and compliance policies in Databricks and Azure
• Hands on experience working on streaming technologies (Kafka, Event Hubs, Kinesis)
• Experience architecting machine learning platforms, advanced analytics workloads and develop and deployment of MLOPS on Databricks

Physical, mental, sensory or environmental demands may be referenced in an attempt to communicate the manner in which this position traditionally is performed

• Expertise with enterprise security models, networking, and cost governance
• Should have experience working with DevOps teams to establish deployment practices using Terraform or similar

Prior experience in leading enterprise Azure Databricks implementations

Optimize Databricks performance (auto-scaling, caching, delta optimization, job tuning, cost optimization)

• For eligible employees, we offer: Flexible work Healthcare including dental, vision, mental health, and well-being programs
• Financial well-being programs such as 401(k) and Employee Share Ownership PlanPaid time off and paid holidays Paid parental leave
• Family building benefits like adoption assistance, surrogacy, and cryopreservationSocial well-being benefits like subsidized back-up child/elder care and tutoring
• The base salary range for the tagged location is $99712 to $168716 /yearly
• This role may be eligible for other compensation including variable compensation, bonus, or commission

Full time regular employees are eligible for paid time off, medical/dental/vision insurance, 401(k), and any other benefits to eligible employees.Note: No amount of pay is considered to be wages or compensation until such amount is earned, vested, and determinable The amount and availability of any bonus, commission, or any other form of compensation that are allocable to a particular employee remains in the Company's sole discretion unless and until paid and may be modified at the Company’s sole discretion, consistent with the law

• The base compensation range for this role in the posted location is$99712 to $168716Company provides compensation range information in accordance with applicable national, state, provincial, and local pay transparency laws
• The base compensation range listed for this position reflects the minimum and maximum target compensation Company, in good faith, believes it may pay for the role at the time of this posting
• The actual compensation offered to any candidate may fall outside of the posted range and will be determined based on multiple factors legally permitted in the applicable jurisdiction
• These may include, but are not limited to: Geographic location, Education and qualifications, Certifications and licenses, Relevant experience and skills, Seniority and performance, Market and business consideration, Internal pay equity

It is not typical for candidates to be hired at or near the top of the posted compensation range.In addition to base salary, this role may be eligible for additional compensation such as variable incentives, bonuses, or commissions, depending on the position and applicable laws

• Company offers a comprehensive, non-negotiable benefits package to all regular, full-time employees

In the U.S. and Canada, available benefits are determined by local policy and eligibility and may include: Paid time off based on employee grade (A-F), defined by policy: Vacation: 12-25 days, depending on grade, Company paid holidays, Personal Days, Sick LeaveMedical, dental, and vision coverage (or provincial healthcare coordination in Canada)Retirement savings plans (e.g., 401(k) in the U.S., RRSP in Canada)Life and disability insurance

• Employee assistance programs
• Other benefits as provided by local policy and eligibility

Important Notice: Compensation (including bonuses, commissions, or other forms of incentive pay) is not considered earned, vested, or payable until it becomes due under the terms of applicable plans or agreements and is subject to Company’s discretion, consistent with applicable laws