AI Engineer
Job Description
Build a shared AI platform that helps teams develop, deploy, and operate production AI applications and agents safely at scale. As a Lead AI Engineer at Wellington Management, you will help shape durable engineering capabilities anchored on Microsoft Azure and Microsoft Foundry, with a focus on security, observability, developer enablement, and operational excellence.
This hybrid role in Boston, MA brings together platform engineering and applied AI: you will design governed, model-agnostic access, evaluation frameworks that improve quality with automated gates, and runtime controls that keep production workloads reliable. You will also treat the platform as a product through automation, Infrastructure as Code, CI/CD, and self-service onboarding.
Responsibilities
- Design and implement shared platform capabilities for model access, agent runtimes, orchestration, gateways, MCP and tool integration, retrieval services, agent/tool registries, observability, evaluations, guardrails, and FinOps controls.
- Create reusable capabilities that enable application and business teams to deliver secure, observable, resilient production solutions via streamlined self-service.
- Establish patterns, frameworks, and reference implementations for single-agent and multi-agent apps using technologies such as Microsoft Agent Framework or LangGraph.
- Develop standards for interoperability and control, including agent-to-agent interoperability, prompts, context/state/memory, human approvals, kill switches, retries, fallback, versioning, SLIs/SLOs, and controlled promotion to production.
- Build a governed, model-agnostic access layer with authentication, identity propagation, routing, quotas, rate limits, failover, usage attribution, policy enforcement, and provider abstraction.
- Support model access across Azure, AWS, and approved on-premises services, including gateway patterns such as Azure API Management in front of Foundry model deployments.
- Create secure MCP servers and gateways, and define hosting and authorization patterns including OAuth, delegated authorization, on-behalf-of identity, token exchange, tool discovery, registration, and allowlisting.
- Partner with Identity, Cybersecurity, and Cloud Engineering teams to implement enterprise identity controls, including per-workload and per-agent identity lifecycle management and Conditional Access with Microsoft Entra ID and Entra Agent ID.
- Implement role-based access control and least privilege, automate app registrations, eliminate secrets, and support cross-cloud federation and entitlement-aware data access.
- Design and operate evaluation frameworks and regression pipelines to measure task success, tool-use correctness, groundedness, safety, and use-case outcomes, using evaluations as automated quality gates for production promotion.
- Instrument agent runs, LLM requests, retrieval, tool calls, latency, errors, quality, policy events, token usage, and cost using OpenTelemetry-based standards where possible.
- Implement guardrails and testing practices including prompt-injection defenses, data-loss protections, red-team testing, audit trails, and agent/model inventories.
- Align platform capabilities with relevant regulatory and model-risk requirements, including NIST AI Risk Management Framework, SR 11-7, and the EU AI Act where applicable.
- Own major platform capabilities end to end: from architecture through production operations, with pragmatic build-versus-buy decisions and rapid evaluation of changing AI technologies.
- Lead technically complex initiatives across AI, Cloud, Security, Data, Architecture, and product engineering teams, and mentor AI and platform engineers through working software, standards, and credible decision-making.
Requirements
- 8+ years of professional software, cloud, or platform engineering experience.
- 3+ years building production machine learning or advanced data platforms.
- 1-2+ years delivering production LLM or agent-based systems.
- Technical leadership for significant production systems.
- Strong hands-on software engineering skills in Python plus at least one additional language: TypeScript, Java, C#, or Go.
- Deep experience across several areas such as LLM APIs, agentic architecture, tool/function calling, RAG, embeddings and vector retrieval, context engineering, orchestration, model routing, observability, and MCP.
- Hands-on experience designing and operating agent evaluations including test datasets, automated and LLM-as-judge scoring, tool-use and trajectory assessment, safety and groundedness checks, and regression gating in CI/CD.
- Strong Microsoft Azure platform engineering depth, including Microsoft Foundry (Foundry Agent Service), model deployments, Azure AI services, API Management, Entra ID, private networking, Key Vault, Application Insights, Azure Monitor, Infrastructure as Code, and CI/CD.
- Experience building platform capabilities other teams consume through APIs, SDKs, self-service infrastructure, developer tooling, runtime controls, telemetry, and production support.
- Strong understanding of distributed systems, API design, reliability, security, identity, networking, and operational risk.
Benefits
- Retirement plan
- Health and wellbeing, including dental and vision coverage
- Pharmacy coverage
- Health savings account
- Flexible spending accounts and commuter program
- Employee assistance program
- Life and disability insurance
- Adoption assistance
- Back-up childcare
- Tuition/CFA reimbursement
- Paid time off, including leave of absence, paid holidays, volunteer, sick, and vacation time
- Discretionary Corporate Bonus and/or Incentives (if eligible)
- Flexible work environment: work from offices 4 days a week with flexibility to work remotely 1 day a week
Technologies
- Microsoft Azure, Microsoft Foundry, Foundry Agent Service
- Microsoft Agent Framework, LangGraph
- Microsoft Entra ID, Entra Agent ID, Azure API Management
- MCP, OAuth, OpenTelemetry
- NIST AI Risk Management Framework, SR 11-7, EU AI Act
- Python, TypeScript, Java, C#, Go
- CI/CD, Infrastructure as Code, policy as code
- Azure AI services, Key Vault, Application Insights, Azure Monitor, Azure AI Search
- Microsoft Purview, Microsoft Fabric, Kubernetes, AKS, Bicep, Terraform, GitHub Actions, Azure DevOps
- AWS