DataJobs.io
← Back to all jobs

Job Description

Health Data Analytics Institute (HDAI) builds AI systems that move from research to production. In this role, you will own the AI/ML backend that connects predictive modeling and production LLM capabilities, helping deliver reliable scoring and clinical-ready AI summaries through a production-grade AWS stack. This hybrid position is based in Dedham, MA, with a salary range of USD 160,367 to 185,457 per year.

What you’ll work on

You will lead the system that scores patients against hundreds of predictive models in production, then drive architecture improvements to address scaling concerns such as memory pressure and compute efficiency (including support for 580+ models). You will also bring model artifacts from Data Science into a production Python inference pipeline, ensuring correct behavior, data validation, and output parity between the R development environment and production scoring.

On the LLM side, you will build and maintain production integrations on Amazon Bedrock (Claude) for AI summaries, PDD, and conversational interfaces. This includes creating and maintaining a prompt lifecycle (design, versioning, testing, and evaluation), implementing token management, validating outputs, handling evidence citation, supporting structured output parsing, and enabling graceful degradation when failures occur.

To support end-to-end delivery, you will build and maintain production Python services on AWS using services including Lambda, DynamoDB, SQS, and S3, working across the existing stack (including FastAPI) as the team migrates services to a serverless architecture.

How quality is ensured

You will build testing and evaluation frameworks for both model inference and LLM outputs. In collaboration with Clinical and Data Science teams, you will define what “correct” means for AI-generated clinical content, write tests for critical paths, and monitor production output quality to catch regressions early.

Collaboration and delivery

You will partner with Data Science on model release planning, validation, and new prompt development. You will also work with Platform Engineering on deployment infrastructure and collaborate across the Product Engineering team on features that span frontend and backend.

Required qualifications

  • 7+ years of experience as a software engineer, ML engineer, or AI engineer building production systems
  • Strong Python proficiency for production service development
  • Experience building and operating services on AWS (Lambda, API Gateway, DynamoDB, SQS, S3) or equivalent cloud platforms
  • Experience taking model artifacts from Data Science and deploying to production, including understanding model scoring and feature engineering, and validating production parity
  • Experience building production LLM applications (not prototypes), including working with model APIs (Bedrock, OpenAI, or similar), prompt architecture design, and handling generative AI failure modes
  • Uses AI tools such as Claude Code or Cursor as a core part of daily workflow
  • Comfort with ambiguity and decision-making with incomplete information as requirements evolve
  • Clear written and verbal communication

Preferred experience

  • Experience across both ML engineering and LLM engineering
  • Experience with Amazon Bedrock or similar managed LLM services
  • Experience with R or familiarity reading R code
  • Experience with model serving at scale (hundreds of models, multi-tenant environments)
  • Healthcare or regulated industry experience
  • Experience building evaluation and testing frameworks for AI system outputs
  • Experience with FastAPI, event-driven architectures (SQS, SNS), or serverless patterns
  • Familiarity with Terraform and infrastructure-as-code

Technologies

  • Python, AWS, Amazon Bedrock, Claude
  • Lambda, DynamoDB, SQS, S3, API Gateway
  • FastAPI, R, Claude Code, Cursor

Benefits

  • 401(k)
  • 401(k) matching
  • Dental insurance
  • Health insurance
  • Health savings account
  • Paid time off
  • Retirement plan
  • Tuition reimbursement
  • Vision insurance

Similar Jobs