Data Engineer – Azure Databricks and Cloud Data Solutions (Safety)

Southern Company is hiring an Azure-focused Data Engineer to support the Operations Data Analytics team with safety data. This onsite role in Birmingham, AL centers on building secure, reusable, high-performing cloud data services on the Azure Databricks lakehouse platform, aligned to enterprise standards. You will help develop and optimize data pipelines and schemas that power analytics and reporting, with strong emphasis on disciplined change management, timely communication, and continuous improvement.

What you’ll do

• Design and develop robust data pipelines to ingest, store, and process data from multiple sources for analytics and reporting.
• Create and maintain logical and physical data models to support efficient querying, analysis, and visualization.
• Build and maintain ETL processes that move data between systems while supporting data quality and consistency.
• Implement and support lakehouse solutions using medallion architecture, including data engineering pipelines and data storage layers.
• Optimize data lake storage, retrieval, performance, and resource utilization for analytical workloads.
• Develop dynamic, metadata-driven frameworks and scalable data ingestion patterns.
• Support streaming datasets and Databricks Delta Live Tables.
• Apply lakehouse performance-tuning techniques such as partitioning, aggregations, and materialized views.
• Architect, develop, implement, integrate, and support cloud technology solutions that meet business partner expectations for quality, cost, and timeliness.
• Collaborate with developers and infrastructure teams to optimize application code, create reusable services, plan capacity, and protect application and data security.
• Serve as the team subject matter expert for data architecture and Databricks.
• Communicate effectively and promptly with customers, team members, and management, building strong peer and business partner relationships.
• Develop project documentation, follow Change Management policies and procedures, and respond quickly and effectively to operational problems.
• Build understanding of business challenges and goals while exhibiting Our Values.

Required qualifications

• Bachelor’s degree required, preferably in Computer Science, Management Information Systems, Engineering, or another technology-related field.
• Experience with Databricks, including notebooks, clusters, and jobs.
• Experience with a major cloud platform, preferably Microsoft Azure, including Cosmos DB, Storage Application Services, and Azure Data Factory.
• Experience architecting data-oriented solutions using cloud data architecture platforms and patterns, preferably Azure.
• Experience implementing lakehouses with medallion architecture, data engineering pipelines, and data storage layers.
• Experience with dynamic metadata-driven frameworks and large-scale data ingestion patterns.
• Experience developing complex logical and physical data models.
• Experience optimizing cloud PaaS resources to follow best practices and reduce cost, plus lakehouse performance tuning using partitioning, aggregations, materialized views, or similar techniques.
• Experience with Agile, DevOps, and CI/CD methodologies.
• Familiarity with Apache Spark, Delta Lake, and MLflow, plus knowledge of structured and unstructured data across databases, data warehouses, and data lakes.
• Knowledge of software development technologies, methodologies, and associated tools.
• Excellent root cause analysis and problem-solving skills.

Technologies you’ll work with

• Microsoft Azure, Azure Databricks, Unity catalog
• Cosmos DB, Storage Application Services, Azure Data Factory
• Notebooks, clusters, jobs; Databricks Delta Live Tables
• Medallion architecture, ETL
• Partitioning, aggregations, materialized views
• Apache Spark, Delta Lake, MLflow
• Agile, DevOps, CI/CD; structured and unstructured data; lakehouse

Benefits

• Competitive base salary
• Annual incentive awards for eligible employees
• Health, welfare and retirement benefits designed to support physical, financial, and emotional/social well-being
• Additional and specific details about total compensation and benefits provided during the hiring process

What success looks like

• Reliable, efficient data pipelines that support analytics and reporting needs.
• Reusable and well-aligned solutions across on-premises and Azure technology platforms.
• Consistent, high-quality data movement through maintainable ETL and lakehouse processes.
• Optimized application code, cloud resources, performance, and capacity utilization.
• Secure data and application solutions supported by complete project documentation and disciplined change management.
• Strong working relationships and timely communication with business partners, technology teams, and management.
• Fast, effective resolution of operational problems and growing team expertise in data architecture and Databricks.

Preferred background

• Hands-on experience with Microsoft Azure data services and cloud PaaS architecture and resources.
• Familiarity with streaming datasets and Databricks Delta Live Tables.
• Strong technical writing skills.

What we value

• Customer focus: Deliver cloud and data solutions that meet or exceed business partner expectations for quality, cost, and timeliness.
• Ownership and responsiveness: Take responsibility for documentation, change management, and rapid resolution of operational problems.
• Collaboration: Communicate effectively and build productive relationships with customers, peers, management, developers, infrastructure teams, and business partners.
• Continuous learning: Become a subject matter expert, stay close to the business, and seek to understand the company’s challenges and goals.
• Enterprise mindset: Design secure, reusable, aligned solutions and exhibit Our Values.