DataJobs.io
← Back to all jobs

Job Description

This hybrid role for a senior data science professional is based in Long Beach, CA, with a focus on building production-grade AI systems. The work centers on Generative AI and LLM solutions, including RAG pipelines, agentic workflows, evaluation practices, and operational controls for observability, security, and privacy.

Role Overview

As a Sr Data Scientist, you will contribute across the full lifecycle of cloud-based LLM applications. Responsibilities include designing retrieval strategies, engineering prompts and function calls, orchestrating agent workflows, and implementing evaluation and monitoring to ensure quality, reliability, and secure handling of sensitive data.

Key Responsibilities

  • Develop Generative AI and Large Language Models (LLMs) solutions
  • Build RAG pipelines and retrieval strategies
  • Apply prompt engineering, including function calling and prompt versioning
  • Design agentic workflows and tool orchestration
  • Perform LLM evaluation using ground truth, including measurements for accuracy, hallucination, faithfulness, and grounding
  • Implement observability for quality, latency, cost, and drift monitoring
  • Ensure security and privacy, including PII handling and RBAC

Required Qualifications

  • 10+ years of experience with Data Scientist roles
  • Strong Python experience with production delivery
  • Hands-on experience with Generative AI and Large Language Models (LLMs)
  • Experience building RAG pipelines and retrieval strategies
  • Experience with prompt engineering, function calling, and prompt versioning
  • Working knowledge of LangChain, LlamaIndex, or equivalent frameworks
  • Experience with agentic workflows and tool orchestration
  • Experience with LLM evaluation methods, including ground truth, accuracy, hallucination measurement, faithfulness, and grounding
  • Observability experience covering quality, latency, cost, and drift monitoring
  • Experience with cloud LLM platforms such as Azure OpenAI, AWS Bedrock, or Vertex AI
  • Security and privacy experience, including PII handling and RBAC
  • Experience with Airflow or Dagster for workflow orchestration
  • Solid machine learning and data science fundamentals
  • Experience with human-reviewed LLM-as-judge evaluation
  • Experience with Jira and Confluence

Technologies

  • Python
  • LangChain
  • LlamaIndex
  • Airflow
  • Dagster
  • Azure OpenAI
  • AWS Bedrock
  • Vertex AI
  • Jira
  • Confluence

Location and Work Arrangement

Long Beach, CA (hybrid)

Compensation

USD 55 - 70 per hourly

Similar Jobs