Data Engineer 5

Capital One is hiring a Data Engineer 5 in New York, NY (onsite) to lead enterprise data technology transformation with cloud-first platforms for AI and the broader data lifecycle.

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support full-stack technical solutions
• Influence developers, data analysts, and data scientists with expertise across machine learning, distributed microservices, lakehouse architecture, and full-stack systems
• Build solutions using Python and Spark, plus open-source relational and NoSQL databases and cloud data warehousing platforms such as Databricks and Snowflake
• Partner with product managers and software engineers to deliver robust cloud-first data solutions that enable impactful experiences for millions of Americans
• Independently design, build, and deliver cloud data solutions and applications with little or no support from supervisors or managers
• Architect and enforce common data engineering design patterns to improve code quality, maintainability, and reusability across platforms and pipelines
• Design and build scalable, resilient, and operationally efficient data pipelines and platforms, ensuring strong performance as data volume and business demands grow
• Serve as an ambassador for data engineering by communicating technical concepts and data outcomes clearly to internal and external stakeholders
• Act as a force-multiplier by balancing hands-on delivery with mentoring and elevating peer and junior engineer capabilities
• Lead end-to-end, large-scale data initiatives, driving critical architectural decisions and evaluating platform choices such as Snowflake vs Databricks based on technical and business requirements
• Stay current with data trends, experiment with new technologies, participate in internal and external technology communities, and mentor others in the data community

Requirements

• Bachelor’s Degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
• 6+ years of experience in application development (internship experience does not apply)
• 4+ years of experience in distributed data
• 4+ years of experience with SQL
• 4+ years programming with at least one: Python, Java, or Scala
• 4+ years designing and developing data pipelines
• 2+ years in data modeling and designing end-to-end data solutions using both relational and non-relational database systems

Technologies

• Python, Spark, Databricks, Snowflake, SQL
• EMR, Glue, AWS, NoSQL, MongoDB, Cassandra, DynamoDB, Redshift
• Airflow, Dagster
• Monte Carlo, Splunk
• Java, Scala

Benefits

• Comprehensive, competitive, and inclusive set of health, financial, and other benefits

Preferred Qualifications

• Master’s Degree in Computer Science or a related field
• 8+ years of experience in data engineering
• 5+ years hands-on experience designing, deploying, and operating data workloads in AWS
• 5+ years experience building or supporting distributed data or compute workloads using EMR, Spark, Glue, or Databricks
• 5+ years experience designing, implementing, and operating real-time or streaming data pipelines
• 5+ years experience working with unstructured or semistructured data using NoSQL databases (MongoDB, Cassandra, DynamoDB)
• 5+ years experience designing and supporting data warehousing solutions (Snowflake, Redshift)
• 4+ years of data modeling experience
• 3+ years experience working in an Agile development environment
• 3+ years experience developing user-centric reusable data products
• 3+ years experience working on data observability (Monte Carlo, Splunk) or data orchestration tools (Airflow, Dagster)

Salary

• New York, NY: USD 250,800 - 286,200 per year
• Performance-based incentive compensation may include cash bonus(es) and/or long term incentives (LTI)