Senior Data Engineer - AI & Analytics Platform - Hybrid

Allianz Life offers a comprehensive benefits package and a hybrid work arrangement in Minneapolis for the Senior Data Engineer - AI Analytics Platform. The role carries a salary range of USD 110,000 to 151,000 per year and centers on building and maintaining data pipelines and platform components on the Azure and Databricks stack to power AI analytics solutions, with a focus on ingestion, data architecture, security, and performance.

Benefits

• A full medical, dental, and vision coverage package
• Flexible spending accounts and health savings accounts
• Tuition reimbursement
• Student loan repayment program
• Generous annual leave
• Competitive 401(k) with company match
• Life insurance
• Onsite health center
• On-site child development center
• On-site fitness facility
• On-site convenience store
• Two cafeterias
• Employee Resource Groups

Responsibilities

• Design, implement, and operate platform components such as data connections, pipelines, Databricks Lakeflow jobs, and related CI/CD pipelines using GitHub Actions and the Databricks Declarative Automation Bundle (DABs)
• Develop templated data engineering project repositories with a custom CLI to streamline usage by the internal platform team; create and manage Databricks DAB infrastructure-as-code repositories as needed
• Ensure high availability, performance, observability, data quality, and cost monitoring through proactive oversight and incident response
• Maintain documentation for the internal platform and for external developers using the platform
• Mentor junior data engineers, review pull requests, and standardize code quality across the platform
• Collaborate with IT, data science, and analytics engineering teams to deliver end-to-end solutions, including provisioning Databricks DAB infrastructure for ingestion and transformation, and supporting DevOps pipelines
• Provide SDLC guidance across CI/CD, release management, and environment promotion processes
• Offer ad hoc deployment support for Databricks Apps space and Azure App Services; Kubernetes experience is preferred
• Evaluate emerging Azure and Databricks services and AI/ML capabilities to drive innovation and continuous improvement
• Contribute to data engineering platform governance, including data ethics, compliance, and security best practices
• Mentor technical teams and promote architectural standards and reusable patterns
• Utilize AI tools and resources, including generative AI, to enhance platform capabilities

Requirements

• 6+ years of experience in a mix of data engineering, DevOps, and application development roles
• Hands-on experience configuring and maintaining Databricks workspaces, ACLs, and related workspace artifacts
• Proficiency with Databricks Lakehouse, Spark, Unity Catalog, and Delta tables
• Experience with DevOps and SDLC practices in a regulated enterprise environment
• Proven ability to create GitHub Actions workflows for promoting code across environments
• Experience building data ingestion and transformation pipelines in PySpark, preferably within Databricks
• Legal authorization to work in the United States without sponsorship now or in the future

Technologies

• Azure
• Databricks
• Databricks Lakeflow
• GitHub Actions
• Databricks Declarative Automation Bundle (DABs)
• Databricks DAB Infrastructure as Code
• Unity Catalog
• Delta tables
• Spark
• PySpark
• Databricks Workspaces
• ACLs
• Azure App Services
• Kubernetes