DataJobs.io
← Back to all jobs

Job Description

The Senior Data Engineer role at CoStar Group focuses on designing, building, and operating reliable, scalable data pipelines and data products within a modern cloud data platform. This is a hands-on position with emphasis on production reliability, performance optimization, and continuous improvement.

Location

Irvine, CA (onsite)

Compensation

USD 128,000 - 190,000 per yearly (Base Compensation: $128,000- $190,000 annually)

Role Responsibilities

  • Design, build, test, and maintain reliable, scalable data pipelines and data products.
  • Develop and optimize data transformations and orchestrations using SQL, Python, dbt, Airflow, Glue, PostgreSQL, and Snowflake.
  • Translate business requirements into maintainable technical designs and production-ready solutions.
  • Implement automated testing, data-quality checks, monitoring, failure handling, and recovery.
  • Optimize large-scale data workloads for performance, scalability, and cost.
  • Troubleshoot production issues and perform root-cause analysis across pipelines and databases.
  • Use Git-based development practices, including code review, CI/CD, and controlled release approaches.
  • Ensure data products are documented, observable, governed, and supported by appropriate metadata and lineage.
  • Reduce technical debt, manual processes, and recurring operational issues through automation and improved design.
  • Collaborate with Product Management, Application Engineering, and business stakeholders.

Required Qualifications

  • 6+ years of professional experience in data engineering or a closely related discipline.
  • Strong Python development skills for data pipelines, automation, integrations, and supporting data applications.
  • Advanced SQL skills and experience working with large-scale datasets.
  • Strong understanding of data modeling, database design, and data transformation patterns.
  • Experience with dbt, including model development, dependency management, testing, incremental processing, and production deployment.
  • Experience with Airflow, including DAG development, orchestration, dependencies, retries, failure handling, scheduling, and production operations.
  • Hands-on PostgreSQL experience, including complex SQL development, database design, indexing, query optimization, and performance tuning.
  • Hands-on Snowflake experience in the context of production data workloads.
  • Experience with AWS data services such as S3, RDS/Aurora, Glue, MWAA, IAM, and CloudWatch.
  • Experience implementing automated data validation and quality controls.
  • Experience with Git, including pull requests, code review, conflict resolution, and CI/CD workflows.
  • Demonstrated ability to troubleshoot complex production pipeline and data issues.
  • Understanding of monitoring, failure recovery, and production support practices.

Technologies

  • SQL, Python, dbt, Airflow, Glue, PostgreSQL, Snowflake
  • AWS S3, RDS/Aurora, MWAA, IAM, CloudWatch
  • Git, CI/CD

Preferred Qualifications

  • Experience with Amazon Aurora PostgreSQL.
  • Experience working with datasets containing hundreds of millions or billions of records.
  • Experience with AWS Glue and Amazon MWAA.
  • Experience with geospatial data and PostGIS.
  • Experience with SQL Server.
  • Experience with DataHub or another metadata, lineage, or governance platform.
  • Experience with Grafana or comparable observability tooling.
  • Experience with database schema migration and version-control frameworks.
  • Experience working within controlled production release and change-management processes.
  • Strong hands-on engineering capability and ownership.
  • Ability to independently deliver complex data engineering solutions.
  • Strong analytical, troubleshooting, and root-cause-analysis skills.
  • Ability to work effectively with large and complex datasets.
  • Strong focus on reliability, performance, scalability, and maintainability.
  • Ability to balance delivery requirements with engineering quality.
  • Strong written and verbal communication.
  • Ability to collaborate effectively with technical and non-technical stakeholders.

Benefits

  • Comprehensive healthcare coverage: Medical / Vision / Dental / Prescription Drug
  • Life, legal, and supplementary insurance
  • Virtual and in person mental health counseling services for individuals and family
  • Commuter and parking benefits
  • 401(K) retirement plan with matching contributions
  • Employee stock purchase plan
  • Paid time off
  • Tuition reimbursement
  • On-site fitness center and/or reimbursed fitness center membership costs (location dependent)
  • Access to CoStar Group’s Employee Resource Groups
  • Complimentary gourmet coffee, tea, hot chocolate, fresh fruit, and other healthy snacks

Additional Information

  • CoStar Group is not able to provide visa sponsorship for this position.
  • CoStar Group is an Equal Employment Opportunity Employer; they maintain a drug-free workplace and perform pre-employment substance abuse testing.

Similar Jobs