Principal Data Engineer

Disney Entertainment and ESPN Product & Technology is hiring a Principal Data Engineer to lead Identity and Device data engineering for marketing measurement and analytics.

Responsibilities

• Define and drive technical strategy for Identity & Device data products in Marketing and Consumer Data, including pipeline architecture, platform patterns, and a modernization roadmap in partnership with onshore squad leaders and enterprise architecture.
• Own end-to-end delivery for complex, high-impact initiatives spanning Identity360, device/consumer identifier datasets, and dependent Marketing data products, including technical approach, sequencing, and engineering quality standards for other engineers.
• Act as principal subject matter expert and escalation point for identity foundational data, covering data modeling, batch and near-real-time processing, performance at scale, and production reliability.
• Participate in an on-call rotation and lead incident response for critical pipelines.
• Set and enforce engineering standards for the Identity & Device team and Marketing Data Engineering, including code quality, CI/CD, observability, and data governance aligned to GDPR, RTBF, and CCPA, along with reusable platform patterns across Databricks, AWS, and orchestration tooling.
• Mentor and develop engineers through design reviews, pairing, and coaching, building depth in big-data engineering, identity domain knowledge, and operational excellence.
• Collaborate cross-functionally with Product, QA, Operations, Privacy/Legal, and peer Marketing data squads (Data Collection, Activation, Acquisition) to align requirements, resolve cross-domain dependencies, and represent technical decisions to stakeholders.

Requirements

• 10+ years of related big data engineering experience, including modeling and building large data pipelines; strong experience with Python, Scala, or Java; strong AWS fundamentals (S3, Lambda, IAM) preferred.
• 5+ years hands-on experience with performance tuning at scale using distributed systems such as Hadoop or Databricks, with query/process tools such as Spark, Presto, and Hive at the petabyte level.
• Experience with at least one major MPP or cloud database technology: Snowflake, Redshift, BigQuery, or Databricks.
• Solid understanding of data modeling and data warehousing standard methodologies and practices.
• Operational experience in a multi-system/services production environment as a senior data engineer and point-of-contact/SME.
• Strong problem-solving with attention to detail, plus excellent analytical and communication skills.
• Experience with data integration and orchestration toolsets (e.g., Airflow) and CI/CD.
• Demonstrated ability with cloud infrastructure technologies, including Terraform.
• Strong understanding of slowly changing dimensions (SCD tables) and building scalable, performant datasets.
• Bachelor’s degree in computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.

Technologies

• Python, Scala, Java
• AWS, S3, Lambda, IAM
• Hadoop, Databricks, Spark, Presto, Hive
• Snowflake, Redshift, BigQuery
• Airflow, CI/CD, Terraform
• GDPR, RTBF, CCPA

Benefits

• Bonus and/or long-term incentive units may be provided as part of the compensation package.
• Full range of medical, financial, and/or other benefits.

Location: New York, NY (onsite)

Compensation: USD 187,900 - 252,000 per year

Experience: 10+ years

PAY TRANSPARENCY: The hiring range for this position in SFO, CA is $187,900 - $252,000 per year and New York is $179,700 - $241,000 per year.

Note: This position is with Disney Entertainment & Sports LLC, which is part of a business we call Disney Entertainment and ESPN Product & Technology.