Sr Databricks Data Engineer

Join Deloitte in Morristown, NJ, onsite, for a Senior Databricks Data Engineer role that combines technical leadership with opportunities to shape enterprise data platforms. The position offers a competitive salary range of USD 116,200 to 229,100 per year and a discretionary annual incentive program. You will design, build, and optimize cloud-based data engineering solutions on Databricks to modernize data platforms, enable analytics and AI, and drive data-driven decision making across the enterprise.

Responsibilities

• Champion Best Practices: Establish, document, and promote best-in-class approaches for data architecture, integration, and modelling.
• Pipeline Ownership: Oversee the design, development, and maintenance of robust data pipelines and data architectures that support large-scale, enterprise data needs.
• Drive Excellence: Initiate and manage efforts to improve data quality, operational efficiency, and process scalability.
• Team and Technology Lead: Evaluate, pilot, and integrate new big data and analytics technologies, ensuring the organization remains at the cutting edge. Lead, coach, and develop teams of data engineers and architects, fostering technical growth and effective project delivery.
• Data Governance: Consult on, design, and implement governance, security, and compliance strategies tailored to modern cloud data ecosystems.
• Communication: Communicate technical concepts and business value to diverse stakeholders, including executives, business leads, and technology teams.
• DevOps and Automation: Oversee the implementation of CI/CD practices with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell for streamlined deployments and operations.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field
• 5+ years of hands-on experience in data engineering with a focus on Databricks on Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP)
• Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
• Experience with data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
• 1+ year leading complex, cross-functional data projects and technical teams, including experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data engineering pipelines, code, and compute resources
• Ability to travel 50%, on average, based on the work you do and the clients and industries/sectors you serve
• Limited immigration sponsorship may be available

Technologies

• Databricks, AWS, Azure, Google Cloud Platform (GCP)
• Delta Lake, Apache Spark, PySpark
• Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows
• Apache Airflow, Unity Catalog, Databricks Lakeflow
• Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell

Benefits

• Discretionary annual incentive program

Preferred

• Master's degree in Computer Science, Engineering, or a related field
• Experience in one or more of AWS, Azure, and GCP cloud ecosystems and associated big data services
• Experience tuning and optimizing performance in Databricks and Apache Spark environments
• Experience with Databricks Lakeflow
• Experience with artificial intelligence and machine learning solutions