DataJobs.io
← Back to all jobs

Job Description

Build and scale infrastructure that powers real-world AI evaluation, from benchmarking systems to production-grade data and model pipelines.

Responsibilities

  • Architect and build the core modeling foundation for Arena Intelligence data and evaluation products
  • Own end-to-end data, model training, and evaluation pipelines
  • Drive a feedback loop and rapid iteration cadence while shipping new features as part of a tight-nit team
  • Research state-of-the-art evaluation methods and contribute to the long-term vision for a centralized, scalable evaluation platform
  • Shape how model benchmarking systems are built, deployed, and continuously improved across data pipelines and inference APIs
  • Partner with researchers, engineers, and product leadership to turn new ideas into reliable systems
  • Improve reproducibility, scale up to new modalities, and strengthen the ability to understand and compare frontier models

Requirements

  • Strong programming skills with the ability to work across the stack in a recommendation system or LLM stack
  • Experience with deep learning, language models, or reward model training
  • Experience working with LLM workflows including fine-tuning, prompt engineering, and function calling
  • Self-motivated approach with willingness to take ownership of tasks
  • Passion for shipping quality products
  • Solid understanding of statistics and tools/methodologies for evaluating uncertainty tailored to the specific product being shipped

Location and Experience

  • Location: San Francisco Bay Area, CA (onsite)
  • Minimum experience: 4 years

Benefits

  • Comprehensive health and wellness benefits: medical, dental, vision, and additional support programs
  • Opportunity to work on cutting-edge AI with a small, mission-driven team
  • Culture emphasizing transparency, trust, and community impact

What We Offer

  • Competitive compensation aligned to market rates for team members
  • Compensation range is listed on the role posting and depends on experience and work location

Similar Jobs