Principal Data Engineer

Abbott Laboratories is hiring a senior individual contributor to lead enterprise data engineering across cross-domain initiatives. In this Principal Data Engineer role, you will define and steward engineering standards and reference architectures, guide high-complexity work from ambiguity through production support, and remain hands-on to validate and improve critical capabilities.

This position is based in Madison, WI (onsite) and serves as an accountable technical leader for integrated outcomes across multiple data domains, with no people-management responsibility.

Key Responsibilities

• Own the technical outcome of assigned cross-domain initiatives, progressing from initial ambiguity through production and lifecycle support.
• Translate enterprise needs into integrated architecture and executable technical plans, decomposing complex work into deliverable increments across domains.
• Coordinate execution across domain Staff Engineers and partner teams, identifying dependencies and architectural risks, and escalating decisions that require business or delivery authority.
• Stay hands-on with prototypes, reference implementations, critical-path development, technical validation, and production problem solving.
• Define, steward, and evolve enterprise data engineering standards, patterns, and reference architectures, driving convergence where inconsistent or duplicative implementations increase enterprise cost, risk, or operational burden.
• Lead architecture for assigned enterprise capabilities, including semantic and metrics layers, canonical data models, and batch, API, event-driven, and streaming patterns supporting analytics, machine learning, and AI.
• Establish enterprise data-contract standards (schemas, service expectations, compatibility, breaking-change policy, and producer-consumer responsibilities) and partner with Platform Engineering to convert recurring needs into shared capabilities.
• Lead architecture and code reviews for high-complexity or cross-domain work, and coach and mentor Staff and Senior Engineers across domains without formal people authority.
• Communicate architectural tradeoffs clearly to engineering, business, and executive stakeholders, and build reusable guidance that increases consistency across Enterprise Data.
• Evaluate architecture tradeoffs across reliability, scalability, performance, security, privacy, operability, adoption, and technical cost, including driving cost-efficient use of compute, storage, streaming, and orchestration.
• Serve as an enterprise technical escalation for incidents involving multiple domains or shared architecture patterns, leading root-cause analysis and preventive changes for recurring issues.
• Design and review architectures handling protected health information to meet applicable security, privacy, lineage, audit, Quality Management System, HIPAA, CLIA, and regulatory requirements.
• Provide technical direction and due diligence for vendor and external-partner solutions, advancing responsible engineering practices including approved AI-assisted development capabilities.
• Ability to work nights and/or weekends, as needed.

Required Qualifications

• Bachelor’s Degree in Data Science, Computer Science, Information Systems, Mathematics, or Engineering.
• Expert-level experience with software development design and development and relevant domain-specific skills (as listed below).
• Experience with Spark on Databricks or comparable platforms; Python, Scala, SQL, and Snowflake.
• Experience building ETL and ELT data pipelines, including batch and event-driven patterns.
• Experience designing and implementing data modeling solutions using relational, dimensional, and/or NoSQL databases.
• Database architecture testing methodology, including execution of test plans, debugging, and testing scripts/tools.
• Experience with open data file and table formats (Parquet, Avro, Delta Lake), and cloud infrastructure/delivery services (AWS, S3, SQS, GitLab CI/CD).
• REST API development experience; familiarity with BI concepts and Tableau performance considerations.
• Agile development tools, including JIRA and Confluence repository use.
• Demonstrated ability to lead through influence across multiple teams and communicate complex technical decisions to senior engineering, business, and executive stakeholders.
• Ability to perform essential duties with or without accommodation.

Technologies

• Databricks, Unity Catalog, Spark, Python, Scala, SQL, Snowflake
• ETL, ELT, Parquet, Avro, Delta Lake
• AWS, S3, SQS, GitLab CI/CD
• REST API, Tableau, JIRA, Confluence
• Kafka, change data capture, NoSQL
• HIPAA, CLIA, FHIR
• GitLab, Azure, Google Cloud Platform

Preferred Qualifications

• Enterprise-scale experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog.
• Experience with Kafka, change data capture, and production event-streaming architectures.
• Experience designing semantic or metrics layers, canonical data models, and governed data products.
• Cloud data architecture experience across AWS, Azure, or Google Cloud Platform.
• Experience in life sciences, diagnostics, or clinical laboratory environments involving protected health information, including HIPAA, CLIA, FDA, or Quality Management System requirements.
• Experience providing technical assessment and architecture direction for vendor and external-partner platforms.

Base Pay

$129,300.00 – $258,700.00 per year.