DataJobs.io
← Back to all jobs

Job Description

Arlo Technologies, Inc. is seeking a Senior Staff Data Engineer to design and scale its data platform. This role focuses on building data architecture, production pipelines, and data governance on Databricks, while partnering across product, analytics, and data science to deliver trusted, secure data products.

What you will do

  • Define and evolve data architecture on the Databricks Lakehouse using medallion (Bronze/Silver/Gold) patterns for both batch and streaming workloads.
  • Design, build, and maintain scalable ETL/ELT pipelines using Python, PySpark, and SQL.
  • Implement and manage Unity Catalog, including catalogs and schemas, access controls, data lineage, auditing, and secure data sharing.
  • Define and enforce data governance practices covering data quality, metadata management, data classification, retention, privacy, and regulatory compliance.
  • Build and maintain CI/CD pipelines for data workflows using Git, GitHub Actions, Azure DevOps, or Jenkins, including Databricks Asset Bundles and automated testing and deployment.
  • Implement data quality checks, monitoring, and alerting to support reliable, trusted datasets.
  • Monitor and optimize Databricks jobs for performance, reliability, and cost.
  • Mentor engineers, lead design reviews, and contribute to the data roadmap.

What you bring

  • 5–6 years of experience in data engineering, including production data pipelines and architectures.
  • Strong hands-on expertise with Databricks including Spark, Delta Lake, and Workflows.
  • Hands-on experience with Unity Catalog, including access control, lineage, and catalog and schema design.
  • Solid understanding of data governance concepts such as data quality, cataloguing, lineage, privacy, and compliance (including GDPR and CCPA).
  • Advanced Python (PySpark) and SQL skills.
  • Solid experience with CI/CD and DataOps practices.
  • Experience with data modelling and a major cloud platform: AWS, Azure, or GCP.
  • Strong communication and cross-functional collaboration skills.
  • A Bachelor’s degree in Computer Science, Engineering, or a related field.

Tech stack

  • Databricks, Spark, Delta Lake, Workflows, Unity Catalog
  • Python, PySpark, SQL
  • Git, GitHub Actions, Azure DevOps, Jenkins, Databricks Asset Bundles
  • AWS, Azure, GCP

Location and pay

  • Location: Milpitas, CA (onsite)
  • Salary: USD 140,000 - 160,000 per year
  • Expected pay range: $140,000–$160,000/year

Compensation

The total compensation package may also include other elements, including bonus, equity, and a full range of benefits.

Similar Jobs