AI Engineer

At Bain’s Coro team, this AI Engineer role focuses on building AI-infused software and data products that leverage LLM-driven features and agentic workflows. The work spans proof-of-concepts, MVP delivery, and scaled deployments, including retrieval and knowledge pipelines, MLOps and GenAIOps, and secure enterprise integration in real client environments.

This position is based onsite in New York, NY and supports end-to-end development across modern LLM stacks and ML systems, from translating client needs into technical requirements to operating production services with reliability, safety, and measurable performance.

Responsibilities

• Design and develop GenAI applications such as copilots, workflow automation, and decision support for commercial teams using modern LLM stacks.
• Implement agentic workflows where they provide clear value, including tool use, multi-step execution, and human-in-the-loop controls, with attention to reliability, safety, and clear failure modes.
• Design and build advanced search, retrieval, and knowledge pipelines across diverse data structures and stores, including hybrid search, vector stores, graph databases / knowledge graphs, and traditional data platforms. Cover indexing strategies, metadata design, relevance tuning and reranking, freshness, caching, access controls, and source attribution.
• Build robust agent capabilities covering context engineering, memory and state management (short-term and long-term), orchestration, routing, and tool integration patterns.
• Integrate solutions into enterprise environments and workflows through APIs, data systems, and collaboration tools, balancing quality, latency, cost, privacy, and adoption.
• Turn ambiguous client inputs into clear technical requirements, tradeoffs, and delivery plans.
• Build ML solutions end-to-end: data preparation, feature engineering, model selection, training, validation and testing, and performance analysis.
• Apply appropriate methods spanning classical ML and deep learning, including sequence, text, and image models when relevant.
• Create reproducible training and evaluation pipelines using versioning, experiment tracking, robust validation, and clear documentation.
• Demonstrate fluency with deep learning concepts including transformer fundamentals and LLM pre-training versus post-training (instruction tuning and preference optimization approaches).
• Write clean, testable, maintainable code and ship AI services across the full SDLC: build, test, deploy, monitor, and iterate.
• Apply MLOps and GenAIOps practices: CI / CD, reproducibility, environment parity, and model / prompt / agent versioning for operational readiness.
• Build evaluation and observability for GenAI and agentic systems using tracing and instrumentation, regression test suites, automated scoring where appropriate, and iteration loops for prompt and policy optimization.
• Design for secure enterprise deployment including access controls, auditability, data handling for sensitive and PII data, and responsible AI guardrails.
• Build reusable components and accelerators such as templates, evaluation harnesses, connectors, and orchestration patterns that scale across client contexts.
• Communicate clearly with technical and non-technical stakeholders, lead working sessions, present recommendations, and write crisp technical documentation.
• Work with Bain consultants to prioritize critical few technical decisions that unlock business value.
• Support proposal shaping and scoping through effort sizing, architecture options, risk assessment, and delivery roadmaps.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
• 3-5+ years of professional AI / ML engineering experience (or equivalent), with strong backend engineering fundamentals.
• Strong proficiency in Python and experience building APIs / services (e.g., REST / gRPC) and integrating with enterprise systems.
• Hands-on experience building LLM-powered applications with delivery considerations including latency, cost, reliability, and security.
• Experience building advanced retrieval / search systems (hybrid retrieval, vector search, reranking) and comfort working across multiple data stores including vector, graph, relational / document / search.
• Experience implementing agentic patterns covering context management, tool integration, orchestration, and memory / state handling, with modern frameworks (e.g., LangGraph, OpenAI Agents SDK, Pydantic AI) or custom agent loops, and strong judgment about when agentic approaches are appropriate.
• Experience creating reusable skills, tools, and services for agent use (including MCP), with schema validation (e.g., Pydantic) to enforce reliable data contracts.
• Strong engineering practices including testing, code review, version control, CI / CD, and performance profiling.
• Experience deploying and operating services on AWS, GCP, and / or Azure, including environment management, reliability, observability, and scaling.
• Experience with Docker and Kubernetes (or equivalent orchestration) and operating services in production, including debugging, performance, and resilience.
• Proven ability to implement security, privacy, and governance requirements including authentication / authorization, access controls, PII / sensitive data handling, and enterprise risk controls.
• Experience training, validating, and testing ML models with strong understanding of overfitting, generalization, and evaluation methodology.
• Practical experience with feature engineering and data preprocessing for real-world datasets.
• Familiarity with a broad set of ML algorithms including classical ML and deep learning, and the ability to choose methods that match business and data constraints.
• Familiarity with deep learning frameworks (e.g., PyTorch / TensorFlow) and ML lifecycle tooling such as experiment tracking and model registry / feature store concepts.
• Proven ability to operate in ambiguity and complexity, manage priorities, and deliver outcomes independently or with a collaborative team.
• Excellent interpersonal and communication skills for explaining technical decisions, tradeoffs, and results to mixed audiences.
• Strong stakeholder management skills and comfort working directly with clients.

Technologies

• English, Python, REST, gRPC
• LLM stacks; LangGraph; OpenAI Agents SDK; Pydantic AI; Pydantic; MCP
• AWS, GCP, Azure
• Docker, Kubernetes
• Vector stores, graph databases, knowledge graphs
• Hybrid retrieval, vector search, reranking
• Transformer fundamentals, instruction tuning, preference optimization approaches
• PyTorch, TensorFlow
• CI / CD, APIs, data pipelines

Benefits

• Medical, dental and vision programs with 100% individual employee premiums.
• Generous paid time off including parental leave, sick leave, and paid holidays.
• Fully vested 401(k) company contribution.
• Paid Life and Long-Term Disability insurance.
• Annual fitness reimbursements.
• 401(k) plan with an annual employer contribution based on years of service (4.5% 401(k) company contribution; increases after 3 years of service; 100% vested upon start date).
• Annual discretionary performance bonus.

Preferred

• MBA, or PhD in a technical field.
• Background in consulting, professional services, or B2B analytics environments.
• Experience working with major AI ecosystem partners on real client deployments.

U.S. Compensation Information

Compensation includes base salary, annual discretionary performance bonus, 401(k) plan with an annual employer contribution based on years of service, and Bain’s best in class benefits package. In Massachusetts, New York, District of Columbia, Georgia, Illinois, Texas, Washington, and California, the good-faith, reasonable annualized full-time salary range is $128,500-$171,500, with placement varying based on factors such as experience, education, licensure/certifications, training, and skill level. This role may also be eligible for other elements of discretionary compensation, and includes a 4.5% 401(k) company contribution that increases after 3 years of service and is 100% vested upon start date.