The Data Engineer 4 (Manager, IC) role is a cloud-first engineering position focused on building and delivering data solutions, while leading technical execution across Agile teams.
Location
Chicago, IL (onsite)
Compensation
USD 179,400 - 204,700 per year
Role Summary
As a Data Engineer 4 (Manager, IC), you will build and deliver cloud-first data solutions and lead engineering efforts across Agile teams. The work includes designing and enforcing data engineering patterns, building scalable pipelines, and implementing security and compliance standards.
Responsibilities
- Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions across full-stack development tools and technologies
- Influence a team of developers, data analysts, and data scientists with experience in machine learning, distributed microservices, lakehouse architecture, and full-stack systems
- Use programming languages including Python and Spark, along with open-source relational and NoSQL databases and cloud data warehousing platforms such as Databricks and Snowflake
- Stay current with data trends by experimenting with and learning new technologies, participating in internal and external technology communities, and mentoring members of the data community
- Partner with product managers and software engineers to deliver robust cloud-first data solutions that support impactful experiences for millions of Americans
- Independently design, build, and deliver cloud data solutions and applications with little or no support from supervisors or managers
- Architect and enforce common data engineering design patterns to support code quality, maintainability, and reusability across data platforms and pipelines
- Design and build data pipelines and platforms with a focus on scalability, resilience, and operational efficiency, ensuring performance under increasing data volume and business demands
- Implement data security standards, including encryption at rest/transit and fine-grained access control, to ensure compliance with data privacy regulations
- Serve as an ambassador for the data engineering team by communicating technical concepts and data outcomes clearly to internal and external stakeholders
Requirements
- Bachelor’s Degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
- At least 4 years of experience in application development (internship experience does not apply)
- At least 2 years of experience in distributed data
- At least 2 years of experience with SQL
- At least 2 years of experience with one programming language (Python, Java, or Scala)
- At least 2 years of experience in data pipeline design and development
- At least 1 year of experience in data modeling and designing end-to-end data solutions using both relational and non-relational database systems
Technologies
- Python
- Spark
- Databricks
- Snowflake
- SQL
- NoSQL databases
- Relational databases
- Agile
- Machine learning
- Distributed microservices
- Lakehouse architecture
- Encryption at rest
- Encryption at transit
- Fine-grained access control
Preferred Qualifications
- 7+ years of experience in application development with demonstrated proficiency in Python, SQL, Scala, or Java
- 4+ years of hands-on experience designing, deploying, and operating data workloads in at least one public cloud environment (AWS, Microsoft Azure, or Google Cloud)
- 4+ years of experience building or supporting distributed data or compute workloads using tools such as EMR, Spark, Glue, or Databricks
- 4+ years of experience designing, implementing, and operating real-time or streaming data pipelines
- 2+ years of experience working on data observability (e.g., Monte Carlo, Splunk) or data orchestration tools (e.g., Airflow, Dagster)
- 4+ years of experience working with unstructured or semistructured data using NoSQL databases (e.g., MongoDB, Cassandra, DynamoDB)
- 4+ years of experience designing and supporting data warehousing solutions (e.g., Snowflake, Redshift)
- 2+ years of experience working in an Agile development environment
- 2+ years of experience developing user-centric reusable data products
Immigration Authorization
Capital One will not sponsor a new applicant for employment authorization, or offer any immigration-related support for this position (including H1B, F-1 OPT, F-1 STEM OPT, F-1 CPT, J-1, TN, E-3, and O-1, or other forms of work authorization that require immigration support from an employer).
