Sr Databricks Data Engineer

Senior Databricks Data Engineer role in Deloitte's AI & Data practice, based in McLean, VA (onsite), focused on designing, building, and optimizing cloud data engineering solutions on Databricks to modernize data platforms and enable analytics and AI across the enterprise.

Responsibilities

• Establish and promote best-in-class data architecture, integration, and modeling practices; document guidelines and share across teams.
• Oversee the design, development, and ongoing maintenance of scalable data pipelines and architectures supporting enterprise data needs.
• Lead initiatives to improve data quality, boost operational efficiency, and scale data processes.
• Evaluate, pilot, and integrate new big data and analytics technologies; coach and develop teams of data engineers and architects to ensure successful project delivery.
• Consult on and implement governance, security, and compliance strategies for modern cloud data ecosystems.
• Communicate complex technical concepts and business value to executives, business leads, and technology teams.
• Guide the implementation of CI/CD pipelines using tools such as Azure DevOps, AWS CodePipeline, Jenkins, TFS, and PowerShell to streamline deployments and operations.
• Provide clear guidance to teammates and stakeholders.

Requirements

• Bachelor's degree in Computer Science, Engineering, or related field.
• 5+ years of hands-on data engineering experience with Databricks on AWS, Azure, or GCP.
• Proficiency with Lakehouse architectures, Apache Spark, Delta Lake, cloud-native storage solutions, and distributed compute platforms.
• Experience in data warehousing (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark.
• At least 1 year leading complex cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of pipelines, code, and compute resources.
• Ability to travel approximately 50 percent based on client needs.
• Limited immigration sponsorship may be available.

Technologies

• Databricks
• Azure DevOps
• AWS Code Pipeline
• Jenkins
• TFS
• PowerShell
• Delta Lake
• Apache Spark
• PySpark
• Delta Live Tables
• Autoloader
• Structured Streaming
• Databricks Workflows
• Unity Catalog
• Apache Airflow
• Databricks Lakeflow
• AWS
• Azure
• GCP

Benefits

• Discretionary annual incentive program, subject to program rules and individual and organizational performance.

The Team

• Deloitte's Core AI & Data practice helps organizations modernize data platforms, strengthen enterprise data foundations, and scale analytics and AI capabilities across the business.
• The team works with clients to architect, engineer, and deploy cloud-based data solutions that improve decision-making, enable innovation, and support large-scale transformation.
• Practitioners collaborate across business and technology functions to solve challenges in data modernization, governance, platform engineering, and insight delivery.

Preferred Qualifications

• Master's degree in Computer Science, Engineering, or related field.
• Experience across AWS, Azure, and GCP cloud ecosystems with associated big data services.
• Proven ability to tune and optimize performance in Databricks and Apache Spark environments.
• Experience with Databricks Lakeflow.
• Experience with artificial intelligence and machine learning solutions.