Machine Learning Engineer
Job Description
Build and scale infrastructure that powers real-world AI evaluation, from benchmarking systems to production-grade data and model pipelines.
Responsibilities
- Architect and build the core modeling foundation for Arena Intelligence data and evaluation products
- Own end-to-end data, model training, and evaluation pipelines
- Drive a feedback loop and rapid iteration cadence while shipping new features as part of a tight-nit team
- Research state-of-the-art evaluation methods and contribute to the long-term vision for a centralized, scalable evaluation platform
- Shape how model benchmarking systems are built, deployed, and continuously improved across data pipelines and inference APIs
- Partner with researchers, engineers, and product leadership to turn new ideas into reliable systems
- Improve reproducibility, scale up to new modalities, and strengthen the ability to understand and compare frontier models
Requirements
- Strong programming skills with the ability to work across the stack in a recommendation system or LLM stack
- Experience with deep learning, language models, or reward model training
- Experience working with LLM workflows including fine-tuning, prompt engineering, and function calling
- Self-motivated approach with willingness to take ownership of tasks
- Passion for shipping quality products
- Solid understanding of statistics and tools/methodologies for evaluating uncertainty tailored to the specific product being shipped
Location and Experience
- Location: San Francisco Bay Area, CA (onsite)
- Minimum experience: 4 years
Benefits
- Comprehensive health and wellness benefits: medical, dental, vision, and additional support programs
- Opportunity to work on cutting-edge AI with a small, mission-driven team
- Culture emphasizing transparency, trust, and community impact
What We Offer
- Competitive compensation aligned to market rates for team members
- Compensation range is listed on the role posting and depends on experience and work location