Sr Databricks Data Engineer
Analytics
Azure
Azure Data Lake Storage
Azure Databricks
Azure Synapse Analytics
Big Data
Bigdata
Cloud
Cloud Operations
Cloud Platforms
Data Analytics
Data Architecture
Data Engineer
Data Governance
Data Integration
Data Lake
Data Pipeline
Data Platform
Data Processing
Data Security
Database
Databricks
Databricks Workflows
Delta Live Tables
DevOps
Engineering
ETL
Spark
SQL
Technical Lead
Job Description
Senior Databricks Data Engineer in Deloitte's AI and Data practice, onsite in Nashville, TN, with a salary range of USD 137,500 to 193,600 per year.
Responsibilities
- Establish and promote leading practices for data architecture, integration, and modelling across projects
- Own the end-to-end design, development, and ongoing maintenance of scalable data pipelines and architectures for enterprise data needs
- Lead initiatives to improve data quality, operational efficiency, and scalability of data processes
- Evaluate, pilot, and integrate emerging big data and analytics technologies; mentor and develop teams of data engineers and architects
- Advise on and implement governance, security, and compliance strategies for cloud data ecosystems
- Convey technical concepts and business value to executives, business leads, and technology teams
- Oversee CI/CD implementations using Azure DevOps, AWS CodePipeline, Jenkins, TFS, and PowerShell to streamline deployments and operations
- Provide clear technical guidance to teammates and stakeholders
Requirements
- Ability to work independently and collaboratively within a team
- Strong written and verbal communication skills
- Keen attention to detail and commitment to high-quality deliverables
- Ability to build and sustain professional relationships
- Experience leading projects or workstreams
- Ability to manage and prioritize multiple tasks in a fast-paced environment
- Strong interpersonal skills and professional demeanor
- Proven ability to meet deadlines
- Bachelor's degree in Computer Science, Engineering, or a related field
- 5+ years of hands-on data engineering with a focus on Databricks on AWS, Azure, or GCP
- Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
- Experience with data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
- 1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of pipelines, code, and compute resources
- Ability to travel up to 50% on average
- Limited immigration sponsorship may be available
- Master's degree in Computer Science, Engineering, or a related field
- Experience across AWS, Azure, and/or GCP cloud ecosystems and related big data services
- Experience tuning and optimizing performance in Databricks and Apache Spark environments
- Experience with Databricks Lakeflow
- Experience with artificial intelligence and machine learning solutions
Technologies
- Databricks platform and Databricks Lakeflow
- Lakehouse architecture and Delta Lake
- Apache Spark and PySpark
- AWS, Microsoft Azure, and Google Cloud Platform (GCP) cloud ecosystems
- Cloud-native databases, storage solutions, and distributed compute platforms
- Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Unity Catalog
- CI/CD tooling: Azure DevOps, AWS CodePipeline, Jenkins, TFS, PowerShell
- Data warehousing and ETL concepts related to 3NF and dimensional modeling
Benefits
- Discretionary annual incentive program
Similar Jobs
