Senior Managing Consultant Data Engineer

Berkeley Research Group is hiring a Senior Managing Consultant Data Engineer (onsite, New York, NY) to lead hands-on data engineering workstreams supporting AI-enabled financial services solutions.

Responsibilities

• Lead data engineering workstreams from discovery through production handover, coordinating technical dependencies, delivery risks, and implementation decisions with client business and technology stakeholders.
• Design and build batch and streaming pipelines in Python and SQL to ingest, transform, and reconcile data from operational systems, APIs, files, and third-party sources.
• Develop ingestion processes for both structured records and unstructured documents, including extraction, metadata capture, exception handling, and traceability back to source material.
• Design warehouse and lakehouse data models for operational reporting, analytics, and AI use cases while preserving business definitions and reconciliation controls.
• Implement data quality checks, lineage, access controls, and PII handling with client security, privacy, and risk teams to support appropriate use of sensitive information.
• Prepare retrieval-ready data by supporting document parsing, chunking, metadata enrichment, embedding pipelines, and access-aware indexing in collaboration with AI engineers.
• Work in the AI Co-lab with domain experts and vendor partners to frame data requirements, compare implementation options, and validate reusable approaches against realistic financial services scenarios.
• Build automated tests, deployment pipelines, monitoring, and recovery procedures to make data products dependable and provide client teams operating guidance.
• Mentor engineers and review code and designs; coordinate with AI, application, change, and risk colleagues to deliver integrated solutions and measure operational impact.
• Capture domain knowledge in reusable schemas, pipeline components, and implementation guides; contribute technical estimates, assumptions, and data dependencies to proposals and next client pursuits.

Requirements

• Minimum experience: 5 years for Managing Consultant level; 7 years for Sr. Managing Consultant level.
• Demonstrated senior, hands-on delivery of production data pipelines and data products; equivalent practical experience may substitute for formal education or certification.
• Strong Python and SQL skills, including ability to design, debug, optimize, and test ingestion and transformation across APIs, databases, files, and documents.
• Experience implementing warehouse or lakehouse models; ability to explain choices around schema design, storage, orchestration, incremental processing, and reconciliation using tools such as Spark and dbt or comparable options.
• Applied data quality, lineage, least-privilege access, and sensitive-data controls in production; ability to translate business and risk requirements into testable engineering controls.
• Experience preparing structured and unstructured data for AI retrieval, including document parsing, chunking, metadata design, indexing, and preservation of source provenance and access permissions.
• Delivered maintainable data services using version control, automated testing, CI/CD, monitoring, and cloud infrastructure; able to diagnose reliability, performance, and cost issues.
• Ability to lead complex technical workstreams, mentor engineers, collaborate with client stakeholders, clarify requirements, communicate trade-offs, and provide credible scope and effort estimates.
• Ability to travel to client sites as needed.

Technologies

• Python
• SQL
• Spark
• dbt
• CI/CD
• Snowflake

Preferred

• Industry experience building operational applications at a leading bank, lender, or financial services institution.
• Financial services experience in consumer lending, commercial lending, or payments (mortgage, credit card, personal loan, or auto lending, including servicing, collections, and default; adjacent banking or wealth and asset management is a plus).
• Experience building retrieval data foundations for production AI applications, including vector or hybrid search, document versioning, and retrieval quality evaluation.
• Experience with change data capture, event-driven processing, or distributed data platforms across cloud and hybrid environments.
• Experience evaluating vendor data capabilities or converting domain expertise into reusable data assets, accelerators, and client implementation patterns.
• Experience implementing the Snowflake platform.

Required (Legal / Work Authorization)

• Candidates must be able to verify their legal right to work in the United States without employer sponsorship now or in the future.

Location and Compensation

• Location: New York, NY (onsite)
• Salary: USD 158,000 - 210,000 per year