Etl Tools Data Jobs

1 job found

Today

Data Engineer 5

Capital One New York, NY

New York, NY

Today

$251k - $286k/yr Full time Backend Developer Apache Airflow Big Data

Apply Now

Job Description

Capital One is hiring a Data Engineer 5 in New York, NY (onsite) to lead enterprise data technology transformation with cloud-first platforms for AI and the broader data lifecycle.

Responsibilities

Collaborate with Agile teams to design, develop, test, implement, and support full-stack technical solutions
Influence developers, data analysts, and data scientists with expertise across machine learning, distributed microservices, lakehouse architecture, and full-stack systems
Build solutions using Python and Spark, plus open-source relational and NoSQL databases and cloud data warehousing platforms such as Databricks and Snowflake
Partner with product managers and software engineers to deliver robust cloud-first data solutions that enable impactful experiences for millions of Americans
Independently design, build, and deliver cloud data solutions and applications with little or no support from supervisors or managers
Architect and enforce common data engineering design patterns to improve code quality, maintainability, and reusability across platforms and pipelines
Design and build scalable, resilient, and operationally efficient data pipelines and platforms, ensuring strong performance as data volume and business demands grow
Serve as an ambassador for data engineering by communicating technical concepts and data outcomes clearly to internal and external stakeholders
Act as a force-multiplier by balancing hands-on delivery with mentoring and elevating peer and junior engineer capabilities
Lead end-to-end, large-scale data initiatives, driving critical architectural decisions and evaluating platform choices such as Snowflake vs Databricks based on technical and business requirements
Stay current with data trends, experiment with new technologies, participate in internal and external technology communities, and mentor others in the data community

Requirements

Bachelor’s Degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
6+ years of experience in application development (internship experience does not apply)
4+ years of experience in distributed data
4+ years of experience with SQL
4+ years programming with at least one: Python, Java, or Scala
4+ years designing and developing data pipelines
2+ years in data modeling and designing end-to-end data solutions using both relational and non-relational database systems

Technologies

Python, Spark, Databricks, Snowflake, SQL
EMR, Glue, AWS, NoSQL, MongoDB, Cassandra, DynamoDB, Redshift
Airflow, Dagster
Monte Carlo, Splunk
Java, Scala

Benefits

Comprehensive, competitive, and inclusive set of health, financial, and other benefits

Preferred Qualifications

Master’s Degree in Computer Science or a related field
8+ years of experience in data engineering
5+ years hands-on experience designing, deploying, and operating data workloads in AWS
5+ years experience building or supporting distributed data or compute workloads using EMR, Spark, Glue, or Databricks
5+ years experience designing, implementing, and operating real-time or streaming data pipelines
5+ years experience working with unstructured or semistructured data using NoSQL databases (MongoDB, Cassandra, DynamoDB)
5+ years experience designing and supporting data warehousing solutions (Snowflake, Redshift)
4+ years of data modeling experience
3+ years experience working in an Agile development environment
3+ years experience developing user-centric reusable data products
3+ years experience working on data observability (Monte Carlo, Splunk) or data orchestration tools (Airflow, Dagster)

Salary

New York, NY: USD 250,800 - 286,200 per year
Performance-based incentive compensation may include cash bonus(es) and/or long term incentives (LTI)

View full job page →

Get Etl Tools Data Jobs alerts

New Etl Tools Data Jobs, daily.