Databricks Engineer & AI Enablement Architect - Data Engineer III
Manager
Artificial Intelligence
Big Data
Bigdata
CI/CD
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lake
Data Lakehouse
Data Lineage
Data Management
Data Pipeline
Data Platform
Data Processing
Data Science
Data Science Ops
Data Security
Data Warehouse
Database
Databases
Databricks
Databricks Lakehouse
Delta Lake
DevOps
Devops Tools
ETL
Informatica
Information Technology (IT)
Integration
Service Principals
Software Development
Software Engineering
Spark
SQL
Job Description
Design and operate secure, scalable Databricks data platforms on Deloitte’s AI & Engineering team, with governance, reliability, and GenAI/LLM production readiness built in.
Responsibilities
- Own governance, architecture, reliability, and day-to-day operation of secure, scalable Databricks environments supporting the Data Aggregation Contract (CDAC)
- Set up workspace governance and Unity Catalog structures, including access controls, service principals, and policies to protect protected health information (PHI) and sensitive healthcare data
- Define medallion architecture standards
- Build reusable ingestion and transformation frameworks for CMS claims and additional healthcare domains, including clinical, provider, quality, and oncology data
- Strengthen data quality and production reliability using validation, reconciliation, lineage, observability, SLA monitoring, dashboards, and proactive alerting
- Establish CI/CD standards, review code, data models, and pipeline designs
- Assess GenAI and LLM use cases for production readiness
Requirements
- Bachelor’s degree
- Ability to obtain and maintain the required clearance for this role
- 4+ years of Databricks engineering experience
- 4+ years with Python, PySpark, and SQL, plus Delta Lake and Medallion Architecture
- 2+ years of experience with databricks architecture
- Ability to travel 10–15% on average, based on work and client/industry needs
- Must be legally authorized to work in the United States without employer sponsorship, now or in the future
Technologies
- Databricks, Unity Catalog
- Python, PySpark, SQL
- Delta Lake, Medallion Architecture
- CI/CD, GenAI, LLM
- service principals
Preferred
- Experience supporting federal healthcare clients, including work with CMS claims, Medicaid/Medicare claims, clinical and provider data, quality measures, or longitudinal patient data
- Databricks certifications such as Certified Data Engineer Professional, Data Analyst Associate, Machine Learning Professional, or Generative AI Engineer Associate
- Experience administering Databricks platforms, including Unity Catalog, workspace administration, cluster policies, service principals, secrets management, and role-based access controls
- Experience implementing Databricks Asset Bundles and Git-based CI/CD across development, test, and production
Team
- Deloitte’s Government & Public Services (GPS) practice focuses on impact across federal, state, and local government clients and public higher education institutions
- Deloitte’s AI & Data offering delivers solutions for designing, developing, and operating data and AI platforms, products, insights, and services
- Support clients in maturing and scaling data, AI, and analytics capabilities
Location
- Washington, DC (onsite)
Salary & Incentive
- Estimated wage range: $113,000 to $188,400 per year
- May be eligible for a discretionary annual incentive program, subject to program rules and factors including individual and organizational performance